hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Subru Krishnan (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (YARN-3666) Federation Intercepting and propagating AM-RM communications (part one: home RM only)
Date Tue, 30 May 2017 20:59:04 GMT

    [ https://issues.apache.org/jira/browse/YARN-3666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16030001#comment-16030001
] 

Subru Krishnan edited comment on YARN-3666 at 5/30/17 8:58 PM:
---------------------------------------------------------------

Thanks [~botong] for the updated patch. It's pretty close, have a couple of minor comments:
  * We have both a ConcurrentHashMap and synchronized access for _containerIdToSubClusterIdMap_,
I feel the latter is redundant.
  * In *cacheAllocatedContainers*, we need a more nuanced approach than simply throwing exception
on encountering containerId clash. We have to think about either reverting all (or selective)
cached containers and/or continue with the remaining containers in the same allocation. Since
this is an extreme corner case, I am fine with dealing with it fully in a separate JIRA.
  * I think you missed my comments on code documentation.


was (Author: subru):
Thanks [~botong] for the updated patch. It's pretty close, have a couple of minor comments:
  * We have both a ConcurrentHashMap and synchronized access for _containerIdToSubClusterIdMap_,
I feel the latter is redundant.
  * I think you missed my comments on code documentation.

> Federation Intercepting and propagating AM-RM communications (part one: home RM only)
> -------------------------------------------------------------------------------------
>
>                 Key: YARN-3666
>                 URL: https://issues.apache.org/jira/browse/YARN-3666
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: nodemanager, resourcemanager
>            Reporter: Kishore Chaliparambil
>            Assignee: Botong Huang
>         Attachments: YARN-3666-YARN-2915.v1.patch, YARN-3666-YARN-2915.v2.patch, YARN-3666-YARN-2915.v3.patch,
YARN-3666-YARN-2915.v4.patch, YARN-3666-YARN-2915.v5.patch, YARN-3666-YARN-2915.v6.patch,
YARN-3666-YARN-2915.v7.patch
>
>
> In order, to support transparent "spanning" of jobs across sub-clusters, all AM-RM communications
are proxied (via YARN-2884).
> This JIRA tracks federation-specific mechanisms that decide how to "split/broadcast"
requests to the RMs and "merge" answers to 
> the AM.
> This the part one jira, which sets up the basic structure, without secondary subclusters.
All requests are forwarded to home subcluster. Part two is in YARN-6511



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message