hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun Suresh (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4496) Improve HA ResourceManager Failover detection on the client
Date Wed, 20 Jan 2016 19:15:39 GMT

    [ https://issues.apache.org/jira/browse/YARN-4496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15109212#comment-15109212

Arun Suresh commented on YARN-4496:

Thanks for the patch [~jianhe] !!

It looks good generally, although, my initial thought process was to actually merge both the
Namenode and RM proxy hierarchies. Currently, it looks like the common ancestor is the {{FailoverProxyProvider}}
class. Unfortunately, for YARN, we have a {{RMFailoverProxyProvider}} which extends it and
adds an {{init()}} method, which in the NN hierarchy, everything is done in the constructor.
I was wondering if we could possibly avoid using the init method and somehow make {{ConfiguredFailoverProxyProvider}}
and {{RequestHedgingProxyProvider}} common enough so that it can be used by both RM and NN..
and avoid some code duplication too.

Thoughts ?

> Improve HA ResourceManager Failover detection on the client
> -----------------------------------------------------------
>                 Key: YARN-4496
>                 URL: https://issues.apache.org/jira/browse/YARN-4496
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: client, resourcemanager
>            Reporter: Arun Suresh
>            Assignee: Jian He
>         Attachments: YARN-4496.1.patch, YARN-4496.2.patch
> HDFS deployments can currently use the {{RequestHedgingProxyProvider}} to improve Namenode
failover detection in the client. It does this by concurrently trying all namenodes and picks
the namenode that returns the fastest with a successful response as the active node.
> It would be useful to have a similar ProxyProvider for the Yarn RM (it can possibly be
done by converging some the class hierarchies to use the same ProxyProvider)
> This would especially be useful for large YARN deployments with multiple standby RMs
where clients will be able to pick the active RM without having to traverse a list of configured

This message was sent by Atlassian JIRA

View raw message