hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hairong Kuang (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-3620) Namenode should synchronously resolve a datanode's network location when the datanode registers
Date Thu, 24 Jul 2008 00:00:31 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-3620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Hairong Kuang updated HADOOP-3620:
----------------------------------

    Attachment: netResolution5.patch

I talked with Raghu and understood his concern is that the number of calls to DNS resolution
might impact the performance of network location resolution performance. From my experiment,
this seems not a big concern. Instead, reducing the number of calls to the script would greatly
improve the resolution performance.

But this new patch reduces all possible calls to DNS resolution. It has all the following
changes:
1. Increase maxArg of ScriptBasedMapping from 20 to 100;
2. CachedDNSToSwitchMap maps IP addresses to the network location;
3. It allows include/exclude host files to contain ip addresses.

> Namenode should synchronously resolve a datanode's network location when the datanode
registers
> -----------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-3620
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3620
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: dfs
>    Affects Versions: 0.18.0
>            Reporter: Hairong Kuang
>            Assignee: Hairong Kuang
>             Fix For: 0.19.0
>
>         Attachments: netResolution.patch, netResolution1.patch, netResolution2.patch,
netResolution3.patch, netResolution4.patch, netResolution5.patch
>
>
> Release 0.18.0 removes the rpc timeout. So the namenode is ok to resolve a datanode's
network location when the datanode registers. This could remove quite a lot of unnecessary
code in both datanode and namenode to handle asynchronous network location resolution and
avoid many potential bugs.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message