hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "E. Sammer" <e...@lifeless.net>
Subject Re: Ubuntu Single Node Tutorial failure. No live or dead nodes.
Date Wed, 10 Feb 2010 20:49:09 GMT
Nick:

It appears that the datanode daemon isn't running.

 > /usr/local/hadoop/bin$ jps
 > 24440 SecondaryNameNode
 > 24626 TaskTracker
 > 24527 JobTracker
 > 24218 NameNode
 > 24725 Jps

There's no process for DataNode. This is the process that is responsible 
for storing blocks. In other words, no data node, no storage. The 
namenode only manages the file system image and metadata.

Check the logs $HADOOP_HOME/logs/*datanode*.log for errors in start up.

Hope this helps.

On 2/10/10 3:42 PM, Nick Klosterman wrote:
> I've been following Michael Noll's Single  node cluster tutorial but am
> unable to run the wordcount example successfully.
>
> It appears that I'm having some sort of problem involving the nodes.
> Using copyFromLocal fails to replicate the data across 1 node.
> When I try to look at the hadoop web interfaces I see that there aren't
> any nodes (alive or dead) listed.
>
> After I start things this is what I get from dfsadmin -report
> /usr/local/hadoop/bin$ ./hadoop dfsadmin -report
> Configured Capacity: 0 (0 KB)
> Present Capacity: 0 (0 KB)
> DFS Remaining: 0 (0 KB)
> DFS Used: 0 (0 KB)
> DFS Used%: %
> Under replicated blocks: 0
> Blocks with corrupt replicas: 0
> Missing blocks: 0
>
> -------------------------------------------------
> Datanodes available: 0 (0 total, 0 dead)
>
>
> Here are the commands I'm entering and the output of them:
>
> /usr/local/hadoop/bin$ ./start-all.sh
> starting namenode, logging to
> /usr/local/hadoop/bin/../logs/hadoop-hadoop-namenode-potr134pc26.out
> localhost: starting datanode, logging to
> /usr/local/hadoop/bin/../logs/hadoop-hadoop-datanode-potr134pc26.out
> localhost: starting secondarynamenode, logging to
> /usr/local/hadoop/bin/../logs/hadoop-hadoop-secondarynamenode-potr134pc26.out
>
> starting jobtracker, logging to
> /usr/local/hadoop/bin/../logs/hadoop-hadoop-jobtracker-potr134pc26.out
> localhost: starting tasktracker, logging to
> /usr/local/hadoop/bin/../logs/hadoop-hadoop-tasktracker-potr134pc26.out
>
> /usr/local/hadoop/bin$ jps
> 24440 SecondaryNameNode
> 24626 TaskTracker
> 24527 JobTracker
> 24218 NameNode
> 24725 Jps
>
> ---> I had all ready created the txtinput directory with ./hadoop dfs
> -mkdir txtinput
>
> /usr/local/hadoop/bin$ ./hadoop dfs -copyFromLocal
> /home/hadoop/Desktop/*.txt txtinput
> 10/02/10 15:29:38 WARN hdfs.DFSClient: DataStreamer Exception:
> org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
> /user/hadoop/txtinput/20417.txt could only be replicated to 0 nodes,
> instead of 1
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1267)
>
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:422)
> at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:508)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:959)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:955)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:953)
>
> at org.apache.hadoop.ipc.Client.call(Client.java:739)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:220)
> at $Proxy0.addBlock(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>
> at $Proxy0.addBlock(Unknown Source)
> at
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:2904)
>
> at
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:2786)
>
> at
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2000(DFSClient.java:2076)
>
> at
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2262)
>
>
> 10/02/10 15:29:38 WARN hdfs.DFSClient: Error Recovery for block null bad
> datanode[0] nodes == null
> 10/02/10 15:29:38 WARN hdfs.DFSClient: Could not get block locations.
> Source file "/user/hadoop/txtinput/20417.txt" - Aborting...
> 10/02/10 15:29:38 WARN hdfs.DFSClient: DataStreamer Exception:
> org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
> /user/hadoop/txtinput/7ldvc10.txt could only be replicated to 0 nodes,
> instead of 1
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1267)
>
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:422)
> at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:508)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:959)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:955)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:953)
>
> at org.apache.hadoop.ipc.Client.call(Client.java:739)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:220)
> at $Proxy0.addBlock(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>
> at $Proxy0.addBlock(Unknown Source)
> at
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:2904)
>
> at
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:2786)
>
> at
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2000(DFSClient.java:2076)
>
> at
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2262)
>
>
> 10/02/10 15:29:38 WARN hdfs.DFSClient: Error Recovery for block null bad
> datanode[0] nodes == null
> 10/02/10 15:29:38 WARN hdfs.DFSClient: Could not get block locations.
> Source file "/user/hadoop/txtinput/7ldvc10.txt" - Aborting...
> copyFromLocal: java.io.IOException: File /user/hadoop/txtinput/20417.txt
> could only be replicated to 0 nodes, instead of 1
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1267)
>
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:422)
> at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:508)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:959)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:955)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:953)
>
> java.io.IOException: File /user/hadoop/txtinput/7ldvc10.txt could only
> be replicated to 0 nodes, instead of 1
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1267)
>
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:422)
> at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:508)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:959)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:955)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:953)
>
> /usr/local/hadoop/bin$ ./hadoop jar ../hadoop-0.20.1-examples.jar
> wordcount txtinput txtoutput
>
> The last command just ends up sitting there doing nothing with no output.
> Any help getting the nodes up and running would be appreciated.
>
> Thanks,
> Nick
>


-- 
Eric Sammer
eric@lifeless.net
http://esammer.blogspot.com

Mime
View raw message