hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daryn Sharp (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-4269) DatanodeManager#registerDatanode rejects all datanode registrations from localhost in single-node developer setup
Date Mon, 22 Apr 2013 22:15:17 GMT

    [ https://issues.apache.org/jira/browse/HDFS-4269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13638500#comment-13638500
] 

Daryn Sharp commented on HDFS-4269:
-----------------------------------

Sorry, was OOO.  Performing the lookups outside of the namespace lock (a very good change
in and of itself!) does not alleviate the performance penalty associated with say lookups
of 4K nodes.

bq.  Requiring a valid reverse DNS lookup for all datanode IP addresses seems like a drastic
solution to a performance problem in an out-of-band management service.

It's bigger than that.  The allow and deny lists allow either hostnames or IP addresses to
be specified.  Let's say you chose to reject or decommission nodes by _hostname_.  There's
a DNS hiccup, DNS outage, packet loss that drop the udp requests, that causes a reverse lookup
to erroneously fail - now that node slips back in and becomes active again!

If you want to allow unresolvable IPs, then you essentially have to ban hostnames in the allow
and deny host lists.

Editing /etc/hosts on the NN should be sufficient.  I don't think you'll encounter client
problems.

                
> DatanodeManager#registerDatanode rejects all datanode registrations from localhost in
single-node developer setup
> -----------------------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-4269
>                 URL: https://issues.apache.org/jira/browse/HDFS-4269
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: namenode
>    Affects Versions: 3.0.0, trunk-win, 2.0.5-beta
>            Reporter: Chris Nauroth
>            Assignee: Chris Nauroth
>             Fix For: 3.0.0
>
>         Attachments: HDFS-4269.1.patch, HDFS-4269.2.patch, HDFS-4269.3.patch
>
>
> HDFS-3990 is a change that optimized some redundant DNS lookups.  As part of that change,
{{DatanodeManager#registerDatanode}} now rejects attempts to register a datanode for which
the name has not been resolved.  Unfortunately, this broke single-node developer setups on
Windows, because Windows does not resolve 127.0.0.1 to "localhost".

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message