Return-Path: X-Original-To: apmail-hadoop-hdfs-issues-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 08EDD1083C for ; Thu, 3 Apr 2014 02:36:30 +0000 (UTC) Received: (qmail 82322 invoked by uid 500); 3 Apr 2014 02:36:17 -0000 Delivered-To: apmail-hadoop-hdfs-issues-archive@hadoop.apache.org Received: (qmail 82235 invoked by uid 500); 3 Apr 2014 02:36:16 -0000 Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hdfs-issues@hadoop.apache.org Delivered-To: mailing list hdfs-issues@hadoop.apache.org Received: (qmail 82221 invoked by uid 99); 3 Apr 2014 02:36:16 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 03 Apr 2014 02:36:16 +0000 Date: Thu, 3 Apr 2014 02:36:16 +0000 (UTC) From: "Tsz Wo Nicholas Sze (JIRA)" To: hdfs-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Resolved] (HDFS-1106) Datanode throwing UnregisteredDatanodeException -- expects itself to serve storage! MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HDFS-1106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze resolved HDFS-1106. --------------------------------------- Resolution: Not a Problem Resolving as Not a Problem. > Datanode throwing UnregisteredDatanodeException -- expects itself to serve storage! > ----------------------------------------------------------------------------------- > > Key: HDFS-1106 > URL: https://issues.apache.org/jira/browse/HDFS-1106 > Project: Hadoop HDFS > Issue Type: Bug > Affects Versions: 0.20.1 > Reporter: Eugene Hung > > We run a large Hadoop cluster used by many different universities. When some DataNodes went down recently, they came back up and then generated this error > message in their datanode logs: > 2010-04-22 16:58:37,314 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.hdfs.protocol.UnregisteredDatanodeException: Data node vm-10-160-4-109:50010 is attempting to report storage ID DS-1884904520-10.160.4.109-50010-1255720271773. Node 10.160.4.109:50010 is expected to serve this storage. > at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getDatanode(FSNamesystem.java:3972) > at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.verifyNodeRegistration(FSNamesystem.java:3937) > at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.registerDatanode(FSNamesystem.java:2052) > at org.apache.hadoop.hdfs.server.namenode.NameNode.register(NameNode.java:735) > at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:508) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:966) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:962) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:396) > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:960) > at org.apache.hadoop.ipc.Client.call(Client.java:740) > at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:220) > at $Proxy4.register(Unknown Source) > at org.apache.hadoop.hdfs.server.datanode.DataNode.register(DataNode.java:544) > at org.apache.hadoop.hdfs.server.datanode.DataNode.runDatanodeDaemon(DataNode.java:1230) > at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1273) > at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:1394) > Note it is correctly expecting itself to serve the data, but throwing an UnregisteredDatanodeException for some reason. This is causing these datanodes to remain "dead" to the namenode. Does anyone know why this is occuring and what we can do to fix it? -- This message was sent by Atlassian JIRA (v6.2#6252)