hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jing Zhao (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6452) ConfiguredFailoverProxyProvider should randomize currentProxyIndex on initialization
Date Wed, 28 May 2014 01:06:02 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14010623#comment-14010623
] 

Jing Zhao commented on HDFS-6452:
---------------------------------

I agree with [~atm]. The NameNode failover is not common in practice, and the administrator
can easily control which NN is the active one while starting the cluster. Given that, to randomize
currentProxyIndex on client initialization will actually increase the number of RPC in normal
cases.

> ConfiguredFailoverProxyProvider should randomize currentProxyIndex on initialization
> ------------------------------------------------------------------------------------
>
>                 Key: HDFS-6452
>                 URL: https://issues.apache.org/jira/browse/HDFS-6452
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: ha, hdfs-client
>    Affects Versions: 2.4.0
>            Reporter: Gera Shegalov
>            Assignee: Gera Shegalov
>
> We observe that the clients iterate proxies in the fixed order. Depending on the order
of namenodes in dfs.ha.namenodes.<nameservice> (e.g. 'nn1,nn2') and the current standby
(nn1), all the clients will hit nn1 first, and then failover to nn2.  Chatting with [~lohit]
we think we can simply select the initial value of {{currentProxyIndex}} randomly, and keep
the logic of {{performFailover}} of iterating from left-to-right. This should halve the unnecessary
load on standby NN.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message