hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "WenJin Ma (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-4450) Duplicate data node on the name node after formatting data node
Date Fri, 01 Feb 2013 02:39:12 GMT

    [ https://issues.apache.org/jira/browse/HDFS-4450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13568403#comment-13568403
] 

WenJin Ma commented on HDFS-4450:
---------------------------------

{quote}
Are you seeing duplicate datanode issue? If so, please post the logs. Per HDFS-3224, as long
as the ipaddr + xfer port remains the same, even if the storage ID changes, duplicate datanode
entry should not occur.
{quote}

1.I deploy a test cluster,have 2 namenodes(active/standby), 3 datanodes.
  Under normal circumstances datanodes such as normal.bmp

2.Fomart datanode dn0 and restart it. hdfs web UI such as exception.bmp.  

3.Namenode not found opteration for remove old datanode. 
2013-02-01 10:11:10,522 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* NameSystem.registerDatanode:
node registration from DatanodeRegistration(10.28.176.234, storageID=DS-685519412-10.28.176.234-50010-1359684666375,
infoPort=50075, ipcPort=50020, storageInfo=lv=-40;cid=CID-fe3b5079-a34a-4912-b8a8-50443d038749;nsid=1321646662;c=0)
storage DS-685519412-10.28.176.234-50010-1359684666375
2013-02-01 10:11:10,555 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: BLOCK*
processReport: Received first block report from 10.28.176.234:50010 after becoming active.
Its block contents are no longer considered stale.
2013-02-01 10:11:10,555 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* processReport: from
DatanodeRegistration(10.28.176.234, storageID=DS-685519412-10.28.176.234-50010-1359684666375,
infoPort=50075, ipcPort=50020, storageInfo=lv=-40;cid=CID-fe3b5079-a34a-4912-b8a8-50443d038749;nsid=1321646662;c=0),
blocks: 0, processing time: 0 msecs

                
> Duplicate data node on the name node after formatting data node
> ---------------------------------------------------------------
>
>                 Key: HDFS-4450
>                 URL: https://issues.apache.org/jira/browse/HDFS-4450
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: namenode
>    Affects Versions: 2.0.2-alpha
>            Reporter: WenJin Ma
>         Attachments: exception.bmp, normal.bmp
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> Duplicate data node on the name node after formatting data node。
> When we registered data node,use nodeReg.getXferPort() to find DatanodeDescriptor.
> {code}
>  DatanodeDescriptor nodeN = host2DatanodeMap.getDatanodeByXferAddr(
>         nodeReg.getIpAddr(), nodeReg.getXferPort());
> {code}
> but add data node use node.getIpAddr().
> {code}
>     /** add node to the map 
>    * return true if the node is added; false otherwise.
>    */
>   boolean add(DatanodeDescriptor node) {
>     hostmapLock.writeLock().lock();
>     try {
>       if (node==null || contains(node)) {
>         return false;
>       }
>       
>       String ipAddr = node.getIpAddr();
>       DatanodeDescriptor[] nodes = map.get(ipAddr);
>       DatanodeDescriptor[] newNodes;
>       if (nodes==null) {
>         newNodes = new DatanodeDescriptor[1];
>         newNodes[0]=node;
>       } else { // rare case: more than one datanode on the host
>         newNodes = new DatanodeDescriptor[nodes.length+1];
>         System.arraycopy(nodes, 0, newNodes, 0, nodes.length);
>         newNodes[nodes.length] = node;
>       }
>       map.put(ipAddr, newNodes);
>       return true;
>     } finally {
>       hostmapLock.writeLock().unlock();
>     }
>   }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message