hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Douglas (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6648) Order of namenodes in ConfiguredFailoverProxyProvider is not defined by order in hdfs-site.xml
Date Fri, 10 Mar 2017 18:42:04 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15905533#comment-15905533

Chris Douglas commented on HDFS-6648:

It looks like the patch is causing {{TestNameNodeRetryCacheMetrics}} to fail; HATestUtil::configureFailoverFs
configures ConfiguredFailoverProxyProvider. I didn't trace the metrics side.

Though randomizing addresses is correct for router-based federation, [~ravwojdyla] was citing
the cost of contacting the Standeby Namenode. Though the patch is as correct as the existing
code, for most deployments, isn't the ordering sometimes useful?

> Order of namenodes in ConfiguredFailoverProxyProvider is not defined by order in hdfs-site.xml
> ----------------------------------------------------------------------------------------------
>                 Key: HDFS-6648
>                 URL: https://issues.apache.org/jira/browse/HDFS-6648
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: ha, hdfs-client
>    Affects Versions: 2.7.0
>            Reporter: Rafal Wojdyla
>         Attachments: HDFS-6648-000.patch
> In org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider, in the
constructor, there's a map <nameservice : < service-id : service-rpc-address > >
(DFSUtil.getHaNnRpcAddresses). It's a LinkedHashMap of HashMaps. The order is kept for _nameservices_.
Then to find active namenode, for nameservice, we get HashMap of <service-id : service-rpc-address
> for requested nameservice (taken from URI request), And for this HashMap we get values
- order of this collection is not strictly defined! In the code: 
> {code}
> Collection<InetSocketAddress> addressesOfNns = addressesInNN.values(); 
> {code}
> And then we put these values (in not defined order) into ArrayList of proxies, and then
in getProxy we start from first proxy in the list and failover to next if needed. 
> It would make sense for ConfiguredFailoverProxyProvider to keep order of proxies/namenodes
defined in hdfs-site.xml.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message