Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Date: Fri, 1 Apr 2016 16:37:25 +0000 (UTC)
From: "Ming Ma (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <JIRA.12953309.1458841595000.113141.1459528645605@Atlassian.JIRA>
In-Reply-To: <JIRA.12953309.1458841595000@Atlassian.JIRA>
References: <JIRA.12953309.1458841595000@Atlassian.JIRA>
 <JIRA.12953309.1458841595923@arcas>
Subject: [jira] [Updated] (HDFS-10208) Addendum for HDFS-9579: to handle the
 case when client machine can't resolve network path
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/HDFS-10208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ming Ma updated HDFS-10208:
---------------------------
    Attachment: HDFS-10208-3.patch

Thanks [~brahmareddy]. I have updated the property description in core-default.xml.

>From the investigation of HDFS-10206, the current way of using Topology tree to compute node distance by reference seems too expensive.  It means it needs to add nodes to the Topology tree first which could become slow as the tree grows. It also means the tree size could grow unbounded. To solve this issue, we can use network path string comparison instead without Topology tree and the extra HashMap in ClientContext.

So in summary, the patch has three improvements:

* Handle the case the client can't resolve network path properly.
* Make the client-side topology resolution optional.
* Use string based comparison for network distance calculation.

> Addendum for HDFS-9579: to handle the case when client machine can't resolve network path
> -----------------------------------------------------------------------------------------
>
>                 Key: HDFS-10208
>                 URL: https://issues.apache.org/jira/browse/HDFS-10208
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: Ming Ma
>            Assignee: Ming Ma
>         Attachments: HDFS-10208-2.patch, HDFS-10208-3.patch, HDFS-10208.patch
>
>
> If DFSClient runs on a machine that can't resolve network path, e.g.{{dnsToSwitchMapping.resolve}} returns null, that will cause exception when it tries to create {{clientNode}}. In such case, there is no need to create {{clientNode}} as null {{clientNode}} means its network distance with any datanode is Integer.MAX_VALUE, which is what we want.
> {noformat}
>     clientNode = new NodeBase(clientHostName,
>         dnsToSwitchMapping.resolve(nodes).get(0));
> {noformat}


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)