hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun Suresh (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-7522) Add application tags manager implementation
Date Tue, 28 Nov 2017 15:11:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-7522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16268877#comment-16268877

Arun Suresh commented on YARN-7522:

Thanks for working on this [~wangda]..
In general, the idea looks fine. Couple of things to consider while fleshing this out:
* We need to figure out at which point in the scheduler / container life cycle are we planning
on calling the addContainer and removeContainer. I propose we do so in a scheduler agnostic
manner - Somewhere in the AbstractYarnScheduler / the AppSchedulingInfo, at the point of allocation
(unfortunately, I don't think AppSchdulingInfo is notified of container release/removal) or
the SchedulingAppAttempt.
* How we are planing on persisting this across RM restarts ? - I am not in favor of pushing
all this information into ZK. This unfortunately means, the tags have to be pushed down to
the NM so they can be retrieved from the NM heartbeats during RM recovery. In that case, we
have to figure out how to deal with the delay from ACQUIRED to RUNNING of a container. The
former is when the RM has allocated and notified the AM and the later is when the NM actually
gets to know about the Container (after the AM has called start container). If we are relying
on the NM to persist this information, we should update the Tag manager only after the the
NM notifies the RM of the running container.

> Add application tags manager implementation
> -------------------------------------------
>                 Key: YARN-7522
>                 URL: https://issues.apache.org/jira/browse/YARN-7522
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Wangda Tan
>            Assignee: Wangda Tan
>         Attachments: YARN-7522.YARN-6592.wip-001.patch
> This is different from YARN-6596, YARN-6596 is targeted to add constraint manager to
store intra/inter application placement constraints. This JIRA is targeted to support storing
maps between container-tags/applications and nodes. This will be required by affinity/anti-affinity
implementation and cardinality.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message