top button
Flag Notify
    Connect to us
      Site Registration

Site Registration

Eclipse plugin for Hadoop2.5.2

+1 vote
418 views

I am new to hadoop. I am using Hadoop2.5.2 and Yarn as MR. I would ask the two ports, M/R(v2) master port and DFS Master port that is to be configured in the Eclipse hadoop plugin view.

Which properties do these ports correspond to in the hadoop configuration files,eg, yarn-site.xml.

posted Dec 9, 2014 by anonymous

Share this question
Facebook Share Button Twitter Share Button LinkedIn Share Button

1 Answer

+1 vote

M/R(v2) master port : Did you mean port for resourcemanager ?

Take a look at ./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml where you can find:yarn.resourcemanager.bind-hostyarn.nodemanager.bind-host

answer Dec 9, 2014 by Honey
Similar Questions
+1 vote
Why can FSDataInputStream.read() only read 2^17 bytes in hadoop2.0?

First I use FileSystem to open a file in hdfs. 

FSDataInputStream m_dis = fs.open(...);

Second, read the data in m_dis to a byte array. 

byte[] inputdata = new byte[m_dis.available()];  //m_dis.available = 47185920
m_dis.read(inputdata, 0, 20 * 1024 * 768 * 3);

The value returned by m_dis.read() is 131072(2^17), so the data after 131072 is missing. It seems that FSDataInputStream use short to manage its data which confused me a lot. The same code run well in hadoop1.2.1.

+1 vote
Can we run mapreduce job from eclipse IDE on fully distributed mode hadoop cluster?

A mapreduce job can be run as jar file from terminal or directly from eclipse IDE. When a job run as jar file from terminal it uses multiple jvm and all resources of cluster. Does the same thing happen when we run from IDE. I have run a job on both and it takes less time on IDE than jar file on terminal.

+1 vote
Using Slider as a default mechanism for HBase on HDP 2.2

We just installed HDP 2.2 through Ambari. We were under the impression that in HDP 2.2., the default deployment mechanism for HBase/Accumulo is through Slider (i.e., they are enabled by default for YARN). However, that does not seem to be the case. Can we choose to install HBase through Slider during HDP installation through Ambari? i.e., was there a customization option that we are missing

If Slider is not the default mechanism for HBase on HDP 2.2, why not?

...