hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bikas Saha (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4278) On AM registration, response should include cluster Nodes report on demanded by registration request.
Date Fri, 23 Oct 2015 04:23:27 GMT

    [ https://issues.apache.org/jira/browse/YARN-4278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14970408#comment-14970408
] 

Bikas Saha commented on YARN-4278:
----------------------------------

Not sure if the alternate to allow an AM to use the ClientRMProtocol was considered in the
discussions. Node reports as well as other data like queue info (which AMs might be also interested
in) would then become available to the AM.

As far as the approach in this jira is concerned, it would be good to check if the AM-RM resync
(after an RM restart) would automatically benefit from this change or whether it would need
to be handled separately.

Any concerns about the size of this data for large clusters? Would it fit easily on the RPC?
Would it cause excessive network usage on the RM nic when many AM's register/resync with it?

> On AM registration, response should  include cluster Nodes report on demanded by registration
request.
> ------------------------------------------------------------------------------------------------------
>
>                 Key: YARN-4278
>                 URL: https://issues.apache.org/jira/browse/YARN-4278
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: api
>            Reporter: Rohith Sharma K S
>            Assignee: Rohith Sharma K S
>
> From the yarn-dev mailing list discussion thread [Thread-1|http://mail-archives.apache.org/mod_mbox/hadoop-yarn-dev/201510.mbox/%3C0EE80F6F7A98A64EBD18F2BE839C91156798AFB3@szxeml512-mbs.china.huawei.com%3E]
[Thread-2|http://mail-archives.apache.org/mod_mbox/hadoop-yarn-dev/201510.mbox/%3C4F7812FC-AB5D-465D-AC89-82473569817E@hortonworks.com%3E]

> Slider required to know about cluster nodes details for providing support for affinity/anti-affinity
on containers.
> Current behavior : During life span of application , updatedNodes are sent in allocate
request only if there are any change like added/removed/'state change' in the nodes. Otherwise
cluster nodes not updated to AM.
> One of the approach thought by [~stevel@apache.org] is while AM registration let response
hold the cluster nodes report



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message