hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dave Latham (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-1106) Datanode throwing UnregisteredDatanodeException -- expects itself to serve storage!
Date Thu, 03 Mar 2011 18:34:37 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002130#comment-13002130
] 

Dave Latham commented on HDFS-1106:
-----------------------------------

I recently saw the same error, and confirmed that the host was not listed in hosts.exclude.
 However, after running hadoop dfsadmin -refreshNodes the problem was solved, so apparently
the node had been in the exclude file previously, and the exclude file had been edited without
refreshing the nodes.  For anyone else here, make sure to try hadoop dfsadmin -refreshNodes

> Datanode throwing UnregisteredDatanodeException -- expects itself to serve storage!
> -----------------------------------------------------------------------------------
>
>                 Key: HDFS-1106
>                 URL: https://issues.apache.org/jira/browse/HDFS-1106
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 0.20.1
>            Reporter: Eugene Hung
>
> We run a large Hadoop cluster used by many different universities.  When some DataNodes
went down recently, they came back up and then generated this error
> message in their datanode logs:
> 2010-04-22 16:58:37,314 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: org.apache.hadoop.ipc.RemoteException:
org.apache.hadoop.hdfs.protocol.UnregisteredDatanodeException: Data node vm-10-160-4-109:50010
is attempting to report storage ID DS-1884904520-10.160.4.109-50010-1255720271773. Node 10.160.4.109:50010
is expected to serve this storage.
> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getDatanode(FSNamesystem.java:3972)
> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.verifyNodeRegistration(FSNamesystem.java:3937)
> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.registerDatanode(FSNamesystem.java:2052)
> 	at org.apache.hadoop.hdfs.server.namenode.NameNode.register(NameNode.java:735)
> 	at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> 	at java.lang.reflect.Method.invoke(Method.java:597)
> 	at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:508)
> 	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:966)
> 	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:962)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:396)
> 	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:960)
> 	at org.apache.hadoop.ipc.Client.call(Client.java:740)
> 	at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:220)
> 	at $Proxy4.register(Unknown Source)
> 	at org.apache.hadoop.hdfs.server.datanode.DataNode.register(DataNode.java:544)
> 	at org.apache.hadoop.hdfs.server.datanode.DataNode.runDatanodeDaemon(DataNode.java:1230)
> 	at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1273)
> 	at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:1394)
> Note it is correctly expecting itself to serve the data, but throwing an UnregisteredDatanodeException
for some reason.  This is causing these datanodes to remain "dead" to the namenode.  Does
anyone know why this is occuring and what we can do to fix it?

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message