hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sean Mackrory (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-11077) RequestHedgingProxyProvider does not behave correctly with more than 3+ NameNodes
Date Mon, 31 Oct 2016 21:14:58 GMT
Sean Mackrory created HDFS-11077:
------------------------------------

             Summary: RequestHedgingProxyProvider does not behave correctly with more than
3+ NameNodes
                 Key: HDFS-11077
                 URL: https://issues.apache.org/jira/browse/HDFS-11077
             Project: Hadoop HDFS
          Issue Type: Bug
            Reporter: Sean Mackrory


RequestHedgingProxyProvider will send its initial request to all NameNodes. It is meant to
then detect the active, and send subsequent requests to the active until it fails over.

It does this by remember the last NameNode to fail, but unless the set of NameNodes minus
the last to fail is a set of size 1, it will continue to use a thread pool and send multiple
requests at a time. If there are 3 or more NameNodes, however, that means a thread pool and
multiple requests will always be sent.

The correct logic in the case of more than 2 NameNodes is probably to use the last known active
and then when it fails, to reset the state, query all of them in parallell, and then use the
one the one that looks like the active.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message