hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantin Shvachko (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-1256) Dfs image loading and edits loading creates multiple instances of DatanodeDescriptor for the same datanode
Date Fri, 13 Apr 2007 01:51:15 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-1256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12488550

Konstantin Shvachko commented on HADOOP-1256:

In my test the image file contains a data-node D0 = <name0, storageID>.
And the edits file has two record [remove D0], [add D1], where D1 = <name1, storageID>.
storageID is the same meaning that the I'm starting the same data-node on different ip addresses/ports.

I start the name-node, and I get an empty edits file and the image containing D1, which means
that the
edits have been applied correctly, everything is as expected.

Then I start data-node D0 and see 2 problems that I believe are related to this issue.
1. The edits file contains 5 add/remove records in it.
There should be just 2: [remove D1], [add D0]
2. The first record in the edits file is [remove D0].
And if I try to restart the name-node it throws UnregisteredDatanodeException exception:

07/04/12 17:39:40 ERROR dfs.NameNode: org.apache.hadoop.dfs.UnregisteredDatanodeException:
Data node <name0> is attempting to report storage ID DS1537505994. Node <name0>
is expected to serve this storage.
    at org.apache.hadoop.dfs.FSNamesystem.getDatanode(FSNamesystem.java:3461)
    at org.apache.hadoop.dfs.FSEditLog.loadFSEdits(FSEditLog.java:311)
    at org.apache.hadoop.dfs.FSImage.loadFSEdits(FSImage.java:672)
    at org.apache.hadoop.dfs.FSImage.loadFSImage(FSImage.java:585)
    at org.apache.hadoop.dfs.FSImage.recoverTransitionRead(FSImage.java:220)
    at org.apache.hadoop.dfs.FSDirectory.loadFSImage(FSDirectory.java:346)
    at org.apache.hadoop.dfs.FSNamesystem.<init>(FSNamesystem.java:251)
    at org.apache.hadoop.dfs.NameNode.init(NameNode.java:173)
    at org.apache.hadoop.dfs.NameNode.<init>(NameNode.java:211)
    at org.apache.hadoop.dfs.NameNode.createNameNode(NameNode.java:820)
    at org.apache.hadoop.dfs.NameNode.main(NameNode.java:828)

I tried the patch it did not fix this problem.

> Dfs image loading and edits loading creates multiple instances of DatanodeDescriptor
for the same datanode
> ----------------------------------------------------------------------------------------------------------
>                 Key: HADOOP-1256
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1256
>             Project: Hadoop
>          Issue Type: Bug
>          Components: dfs
>    Affects Versions: 0.12.3
>            Reporter: Hairong Kuang
>             Fix For: 0.13.0
>         Attachments: nodeMap.patch
> This leads to multiple instances of DatanodeDescriptors for the same datanode stored
in Host2DatanodeMap.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message