hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sisu Xi <xis...@gmail.com>
Subject Re: hadoop multinode, only master node doing the work
Date Sun, 13 Jul 2014 19:15:45 GMT
Hi, Ravi:

Yes! you are right! I did not realize there is a data dir on the slave node
as well. Thanks very much!
After deleting it and restart, now I can see the slave node.

However, when I ran the pi program (even from the slave node), still only
the master node is doing the work. The command I am using is:
xisisu@slave-01:/usr/local/hadoop$ hadoop jar
share/hadoop/mapreduce/hadoop-mapreduce-examples-2.2.0.jar pi 100 10000000

Attached is the output for hadoop dfsadmin -report

xisisu@slave-01:/usr/local/hadoop$ hadoop dfsadmin -report
DEPRECATED: Use of this script to execute hdfs command is deprecated.
Instead use the hdfs command for it.

OpenJDK 64-Bit Server VM warning: You have loaded library
/usr/local/hadoop/lib/native/libhadoop.so.1.0.0 which might have disabled
stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c
<libfile>', or link it with '-z noexecstack'.
14/07/13 14:10:34 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
Configured Capacity: 28137644032 (26.21 GB)
Present Capacity: 16888445354 (15.73 GB)
DFS Remaining: 16887668736 (15.73 GB)
DFS Used: 776618 (758.42 KB)
DFS Used%: 0.00%
Under replicated blocks: 0
Blocks with corrupt replicas: 0
Missing blocks: 0

Datanodes available: 2 (2 total, 0 dead)

Live datanodes:
Name: (slave-01)
Hostname: slave-01
Decommission Status : Normal
Configured Capacity: 14068822016 (13.10 GB)
DFS Used: 388309 (379.21 KB)
Non DFS Used: 5623800619 (5.24 GB)
DFS Remaining: 8444633088 (7.86 GB)
DFS Used%: 0.00%
DFS Remaining%: 60.02%
Last contact: Sun Jul 13 14:10:34 CDT 2014

Name: (master)
Hostname: master
Decommission Status : Normal
Configured Capacity: 14068822016 (13.10 GB)
DFS Used: 388309 (379.21 KB)
Non DFS Used: 5625398059 (5.24 GB)
DFS Remaining: 8443035648 (7.86 GB)
DFS Used%: 0.00%
DFS Remaining%: 60.01%
Last contact: Sun Jul 13 14:10:34 CDT 2014


On Sun, Jul 13, 2014 at 2:03 PM, Ravi Mutyala <ravi@hortonworks.com> wrote:

> Try cleaning up /home/xisisu/mydata/hdfs/datanode on the slave nodes
> andvstart  'datanode' on that again. That should work. If it doesn't, send
> the log message.
> Also see if nodemanagers are running on slaves nodes. If everything is
> running on master, then its not just HDFS issue, its also yarn nodemanagers
> not running.
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.


*Sisu Xi, PhD Candidate*

Department of Computer Science and Engineering
Campus Box 1045
Washington University in St. Louis
One Brookings Drive
St. Louis, MO 63130

View raw message