hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kilaru, Sambaiah" <Sambaiah_Kil...@intuit.com>
Subject Re: hadoop multinode, only master node doing the work
Date Sun, 13 Jul 2014 06:36:48 GMT
HI Sisu Xi,

On the master node can you check
Hadoop dfsadmin –report
And listing all the slave nodes or you can check master URL and it should all datanodes listed
as slave nodes.
Check for RM UI and slave nodes listed there also.

Thanks,
Sam

From: Sisu Xi <xisisu@gmail.com<mailto:xisisu@gmail.com>>
Reply-To: "user@hadoop.apache.org<mailto:user@hadoop.apache.org>" <user@hadoop.apache.org<mailto:user@hadoop.apache.org>>
Date: Sunday, July 13, 2014 at 11:28 AM
To: "user@hadoop.apache.org<mailto:user@hadoop.apache.org>" <user@hadoop.apache.org<mailto:user@hadoop.apache.org>>
Subject: hadoop multinode, only master node doing the work

Hi, all:

I am new to hadoop. I followed the tutorial on
http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-multi-node-cluster/

and installed hadoop 2.2.0 on two 4-core Ubuntu 12.04 machines.

I can start the pi program, however, only the master node is doing the work (I checked the
top on each machine).
Seems the two nodes are configured correctly, because I can start  the program in the slave
node as well, and still only the master node is doing the actual work.
I have tried different number of mappers for the pi program, and the results is the same.

Is there anything else I can check?

In the end is my configure file on each host.

Thanks very much!

Sisu

---------yarn-site.xml-------

<property>
  <name>yarn.nodemanager.aux-services</name>
  <value>mapreduce_shuffle</value>
</property>


<property>
  <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
  <value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>


<property>
  <name>yarn.resourcemanager.address</name>
  <value>master:8032</value>
</property>

---------------hdfs-site.xml--------------------

<property>
  <name>dfs.replication</name>
  <value>2</value>
</property>


<property>
  <name>dfs.namenode.name.dir</name>
  <value>file:/home/xisisu/mydata/hdfs/namenode</value>
</property>


<property>
  <name>dfs.datanode.data.dir</name>
  <value>file:/home/xisisu/mydata/hdfs/datanode</value>
</property>


-------------core-site.xml-------------

<property>
<name>fs.default.name<http://fs.default.name></name>
<value>hdfs://master:9000</value>
</property>

------------------mapred-site.xml-----------------

<property>
  <name>mapreduce.framework.name<http://mapreduce.framework.name></name>
  <value>yarn</value>
</property>

<property>
  <name>mapred.job.tracker</name>
  <value>master:54311</value>
  <description>The host and port that the MapReduce job tracker runs
  at.  If "local", then jobs are run in-process as a single map
  and reduce task.
  </description>
</property>




--
[https://dl.dropboxusercontent.com/u/268069/Sisu_Contact.jpeg]

Sisu Xi, PhD Candidate

http://www.cse.wustl.edu/~xis/
Department of Computer Science and Engineering
Campus Box 1045
Washington University in St. Louis
One Brookings Drive
St. Louis, MO 63130

Mime
View raw message