Mailing-List: contact core-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: core-dev@hadoop.apache.org
Message-ID: <1802647638.1216857631722.JavaMail.jira@brutus>
Date: Wed, 23 Jul 2008 17:00:31 -0700 (PDT)
From: "Hairong Kuang (JIRA)" <jira@apache.org>
To: core-dev@hadoop.apache.org
Subject: [jira] Updated: (HADOOP-3620) Namenode should synchronously resolve
 a datanode's network location when the datanode registers
In-Reply-To: <342581703.1214245125137.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/HADOOP-3620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hairong Kuang updated HADOOP-3620:
----------------------------------

    Attachment: netResolution5.patch

I talked with Raghu and understood his concern is that the number of calls to DNS resolution might impact the performance of network location resolution performance. From my experiment, this seems not a big concern. Instead, reducing the number of calls to the script would greatly improve the resolution performance.

But this new patch reduces all possible calls to DNS resolution. It has all the following changes:
1. Increase maxArg of ScriptBasedMapping from 20 to 100;
2. CachedDNSToSwitchMap maps IP addresses to the network location;
3. It allows include/exclude host files to contain ip addresses.

> Namenode should synchronously resolve a datanode's network location when the datanode registers
> -----------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-3620
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3620
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: dfs
>    Affects Versions: 0.18.0
>            Reporter: Hairong Kuang
>            Assignee: Hairong Kuang
>             Fix For: 0.19.0
>
>         Attachments: netResolution.patch, netResolution1.patch, netResolution2.patch, netResolution3.patch, netResolution4.patch, netResolution5.patch
>
>
> Release 0.18.0 removes the rpc timeout. So the namenode is ok to resolve a datanode's network location when the datanode registers. This could remove quite a lot of unnecessary code in both datanode and namenode to handle asynchronous network location resolution and avoid many potential bugs.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.