hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jagane Sundar (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-4269) DatanodeManager#registerDatanode rejects all datanode registrations from localhost in single-node developer setup
Date Mon, 15 Apr 2013 04:48:17 GMT

    [ https://issues.apache.org/jira/browse/HDFS-4269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13631498#comment-13631498

Jagane Sundar commented on HDFS-4269:

Thanks for your explanation, Chris.

The performance problem you mention is not in the data path or the metadata path of HDFS,
Requiring a valid reverse DNS lookup for all datanode IP addresses seems like a drastic solution
to a performance problem in an out-of-band management service.

I am going to sound like the crusty curmudgeon that I am when I say this: For time immemorial,
TCP services have worked with raw IP addresses that do not have valid reverse lookups. I think
that HDFS should work with raw IP addresses.

Editing /etc/hosts or creating a new DNS server are not solutions. The client machines now
need to know of this new DNS mapping. Forcing users to add the new DNS server to their TCP
stack's configuration, or asking them to edit their /etc/hosts or their C:\windows\system32\drivers\etc\hosts
(editable only by programs running as administrator) is not practical.

I see one possible solution to this: Enhance the web UI to use some AJAX magic and display
IP addresses first, then switch to Hostnames as the reverse lookups in a server thread succeed.

In lieu of this, can we at least add a hdfs-site.xml config option that turns off this check?
> DatanodeManager#registerDatanode rejects all datanode registrations from localhost in
single-node developer setup
> -----------------------------------------------------------------------------------------------------------------
>                 Key: HDFS-4269
>                 URL: https://issues.apache.org/jira/browse/HDFS-4269
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: namenode
>    Affects Versions: 3.0.0, trunk-win, 2.0.5-beta
>            Reporter: Chris Nauroth
>            Assignee: Chris Nauroth
>             Fix For: 3.0.0
>         Attachments: HDFS-4269.1.patch, HDFS-4269.2.patch, HDFS-4269.3.patch
> HDFS-3990 is a change that optimized some redundant DNS lookups.  As part of that change,
{{DatanodeManager#registerDatanode}} now rejects attempts to register a datanode for which
the name has not been resolved.  Unfortunately, this broke single-node developer setups on
Windows, because Windows does not resolve to "localhost".

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message