hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun Suresh (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-7653) Node group support for AllocationTagsManager
Date Fri, 22 Dec 2017 02:12:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-7653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16300873#comment-16300873

Arun Suresh commented on YARN-7653:

Thanks for the patch [~pgaref].
It looks mostly good. One thing I did notice with the original patch is how do we handle Node
removals. Currently tags are removed only when a container is completed. What if a node goes
down ? Would we ever need a tag -> nodes mapping ?
I assume in the algorithm, you would generally iterate over nodes/racks and get the cardinality
- in which case, the issue above is somewhat moot (since you will probably never ask the tags
manager to "give me all nodes with this tag"). Although, might be nice to purge entries from
the maps once in a while. We can do this is in a follow-up JIRA if required.

> Node group support for AllocationTagsManager
> --------------------------------------------
>                 Key: YARN-7653
>                 URL: https://issues.apache.org/jira/browse/YARN-7653
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Panagiotis Garefalakis
>            Assignee: Panagiotis Garefalakis
>         Attachments: YARN-7653-YARN-6592.001.patch, YARN-7653-YARN-6592.002.patch
> AllocationTagsManager currently supports node and cluster-wide tag cardinality retrieval.
> If we want to support arbitrary node-groups/scopes for our placement constraints TagsManager
should be extended to provide such functionality.
> As a first step we need to support RACK scope cardinality retrieval (as defined in our
> i.e. how many "spark" containers are currently running on "RACK-1"

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message