hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Subru Krishnan (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-3672) Create Facade for Federation State and Policy Store
Date Mon, 15 Aug 2016 23:13:21 GMT

     [ https://issues.apache.org/jira/browse/YARN-3672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Subru Krishnan updated YARN-3672:
---------------------------------
    Attachment: YARN-3672-YARN-2915-v4.patch

Thanks [~jianhe] for your feedback. 

bq. How will the RM failover play together with YARN-3673 ? Let’s say subCluster1(RM1, RM2),
subCluster2(RM3, RM4). Looks like the implementation will ignore cluster intra-failover and
do cluster inter-failover only ?

The implementation will handle only cluster intra-failover as the RM failover proxy in YARN-3673
will be seeded based on _subClusterId_. The information on the StateStore will get updated
as part of RM active services initialization (YARN-3671). In your example, the RM failover
proxy will be a _connection_ to subCluster1 which will initially point to say RM1 which is
the current primary. Suppose RM1 fails over to RM2, RM2 will now heartbeat the StateStore
against subCluster1 and we will auto-update the proxy to connect to RM2 (by querying getSubClusterInfo(subCluster1)
on the Facade).
The cluster inter-failover is determined by the policies (YARN-5323) as that defines how a
queue spans multiple sub-clusters and the {{Router/AMRMProxy}} will create a RM failover proxy
per subCluster in the policy.
Makes sense?

bq. question for such API. It asks for a specific subCluster info, do we still need the filterInactiveSubClusters
flag ? Even if it’s required, the behavior for the if/else is inconsistent, the if case
is honoring the flag, while the else doesn’t. 

Good catch. I have removed filterInactiveSubClusters from getSubClusterInfo.

bq. I think we should not reuse these two configs for retry, the default value of both is
zero. 

Valid point. I was trying to reuse existing configs as we have to add a few for Federation
on top of the too many existing ones. I looked at the {{RMProxy}} and have replaced them with
better fitting ones.


I have updated the patch (v4) accordingly.

> Create Facade for Federation State and Policy Store
> ---------------------------------------------------
>
>                 Key: YARN-3672
>                 URL: https://issues.apache.org/jira/browse/YARN-3672
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: nodemanager, resourcemanager
>            Reporter: Subru Krishnan
>            Assignee: Subru Krishnan
>         Attachments: YARN-3672-YARN-2915-v1.patch, YARN-3672-YARN-2915-v2.patch, YARN-3672-YARN-2915-v3.patch,
YARN-3672-YARN-2915-v4.patch
>
>
> This JIRA proposes creating a facade for Federation State and Policy Store to simply
access and have a common place for cache management etc that can be used by both Router &
AMRMProxy



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message