hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eli Collins (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3224) Bug in check for DN re-registration with different storage ID
Date Tue, 09 Oct 2012 15:32:03 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472466#comment-13472466

Eli Collins commented on HDFS-3224:

bq. I don't think it matters whether host or ip is used, it all about adding the port. Since
the rest of the code is expecting the nodes to be keyed on host, using host:port is the minimal
and least risky change.

Just using the IP is the minimal change. Checking the port supports mini clusters but doesn't
work if you use an ephemeral port (eg "" for dfs.datanode.address) because the DN
will re-register with a different port and getDatanodeByXferAddr won't find it. How about
adding a warn if we find an entry for the IP but do not match any of the transfer port?

@Daryn, since getDatanodeByXferAddr lives in Host2NodesMap it seems OK that it use its implementation
details. If we decide DNs are uniquely identified by xferAddr:xferPort then we could key Host2NodesMap
on this rather than just the IP, which would be cleaner.
> Bug in check for DN re-registration with different storage ID
> -------------------------------------------------------------
>                 Key: HDFS-3224
>                 URL: https://issues.apache.org/jira/browse/HDFS-3224
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: Eli Collins
>            Assignee: Jason Lowe
>            Priority: Minor
>         Attachments: HDFS-3224.patch, HDFS-3224.patch
> DatanodeManager#registerDatanode checks the host to node map using an IP:port key, however
the map is keyed on IP, so this check will always fail. It's performing the check to determine
if a DN with the same IP and storage ID has already registered, and if so to remove this DN
from the map and indicate that eg it's no longer hosting these blocks. This bug has been here

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message