Mailing-List: contact core-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: core-dev@hadoop.apache.org
Message-ID: <262596.1216665691728.JavaMail.jira@brutus>
Date: Mon, 21 Jul 2008 11:41:31 -0700 (PDT)
From: "Raghu Angadi (JIRA)" <jira@apache.org>
To: core-dev@hadoop.apache.org
Subject: [jira] Commented: (HADOOP-3620) Namenode should synchronously
 resolve a datanode's network location when the datanode registers
In-Reply-To: <342581703.1214245125137.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HADOOP-3620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12615355#action_12615355 ] 

Raghu Angadi commented on HADOOP-3620:
--------------------------------------

> So network resolution in the front could be an overhead. 
this may not be a problem since a DataNode would not re-register unless there is a real problem/bug. Not sure if we need to optimize that. Even if we want to, then we can make 'internalRegisterDatanode()' throw an exception to indicate that netwo needs to be resolved before calling it. 

I think doing this way will simplify the code and patch even further.

> Pre-resolving should help since it resolves network locations in batch and therefore reducing the number of calls to the rack script dramatically.
Only if the script can do the resolutions in parallel. Does not default script make use of this? Also there are 2 DNS resolutions done for each host serially inside namenode to 'normalize' the host names, right? Also many installations may not specify include hosts.

> Namenode should synchronously resolve a datanode's network location when the datanode registers
> -----------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-3620
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3620
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: dfs
>    Affects Versions: 0.18.0
>            Reporter: Hairong Kuang
>            Assignee: Hairong Kuang
>             Fix For: 0.19.0
>
>         Attachments: netResolution.patch, netResolution1.patch, netResolution2.patch, netResolution3.patch, netResolution4.patch
>
>
> Release 0.18.0 removes the rpc timeout. So the namenode is ok to resolve a datanode's network location when the datanode registers. This could remove quite a lot of unnecessary code in both datanode and namenode to handle asynchronous network location resolution and avoid many potential bugs.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.