hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sangjin Lee (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-10208) Addendum for HDFS-9579: to handle the case when client machine can't resolve network path
Date Tue, 12 Apr 2016 16:19:25 GMT

    [ https://issues.apache.org/jira/browse/HDFS-10208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15237483#comment-15237483
] 

Sangjin Lee commented on HDFS-10208:
------------------------------------

Thanks [~mingma] for the patch!

The default value for the new config is false, meaning we want to disable the feature for
HDFS-9579 by default. Is that desirable? I would think that we should be able to enable it
by default unless people want to turn it off. I'd like to hear your thought on the default...

I see that the lookup map in {{ClientContext}} was removed as part of this patch. However,
if {{getNetworkDistance()}} is called repeatedly on the same datanodes, there is definitely
a benefit in caching the result of the computation. Is the concern for removing the {{nodeToDistance}}
map about its size?


> Addendum for HDFS-9579: to handle the case when client machine can't resolve network
path
> -----------------------------------------------------------------------------------------
>
>                 Key: HDFS-10208
>                 URL: https://issues.apache.org/jira/browse/HDFS-10208
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: Ming Ma
>            Assignee: Ming Ma
>         Attachments: HDFS-10208-2.patch, HDFS-10208-3.patch, HDFS-10208-4.patch, HDFS-10208.patch
>
>
> If DFSClient runs on a machine that can't resolve network path, {{DNSToSwitchMapping}}
will return {{DEFAULT_RACK}}. In addition, if somehow {{dnsToSwitchMapping.resolve}} returns
null, that will cause exception when it tries to create {{clientNode}}. In either case, there
is no need to create {{clientNode}} and we should treat its network distance with any datanode
as Integer.MAX_VALUE.
> {noformat}
>     clientNode = new NodeBase(clientHostName,
>         dnsToSwitchMapping.resolve(nodes).get(0));
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message