hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Raghu Angadi (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3620) Namenode should synchronously resolve a datanode's network location when the datanode registers
Date Mon, 21 Jul 2008 18:41:31 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12615355#action_12615355
] 

Raghu Angadi commented on HADOOP-3620:
--------------------------------------

> So network resolution in the front could be an overhead. 
this may not be a problem since a DataNode would not re-register unless there is a real problem/bug.
Not sure if we need to optimize that. Even if we want to, then we can make 'internalRegisterDatanode()'
throw an exception to indicate that netwo needs to be resolved before calling it. 

I think doing this way will simplify the code and patch even further.

> Pre-resolving should help since it resolves network locations in batch and therefore
reducing the number of calls to the rack script dramatically.
Only if the script can do the resolutions in parallel. Does not default script make use of
this? Also there are 2 DNS resolutions done for each host serially inside namenode to 'normalize'
the host names, right? Also many installations may not specify include hosts.

> Namenode should synchronously resolve a datanode's network location when the datanode
registers
> -----------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-3620
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3620
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: dfs
>    Affects Versions: 0.18.0
>            Reporter: Hairong Kuang
>            Assignee: Hairong Kuang
>             Fix For: 0.19.0
>
>         Attachments: netResolution.patch, netResolution1.patch, netResolution2.patch,
netResolution3.patch, netResolution4.patch
>
>
> Release 0.18.0 removes the rpc timeout. So the namenode is ok to resolve a datanode's
network location when the datanode registers. This could remove quite a lot of unnecessary
code in both datanode and namenode to handle asynchronous network location resolution and
avoid many potential bugs.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message