hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-7522) Add application tags manager implementation
Date Fri, 01 Dec 2017 18:30:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-7522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16274746#comment-16274746

Wangda Tan commented on YARN-7522:

Thanks [~sunilg] for your suggestions.

For 1).  I suggest to not RMNode for performance consideration. Just like node labels manager.
we can fully control the data structure and get constraint without locking RMNode.
For 2).  You're correct for memory cost of each app, it is still an order of #containers belongs
to the app. I think it is not too bad since we already stored the similar size of information
in {{SchedulerApplicationAttempt}} and Preemption policy.  
bq. getCardinality does a loop to compute numValues? Could globalMapping to be improved to
keep a sum also?
I was thinking about this before, however, I'm not sure what is the most popular usage of
cardinality function. In anti-affinity case it might be max, in affinity case it might be
min. In cardinality between apps case, it can be varies. Precomputing and store these values
might be expensive and unnecessary in some cases. Since current computation is an order of
#tags in one node. I think it should be fine at least in the short term. Let's see how it
goes and optimize it once we have more input.

> Add application tags manager implementation
> -------------------------------------------
>                 Key: YARN-7522
>                 URL: https://issues.apache.org/jira/browse/YARN-7522
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Wangda Tan
>            Assignee: Wangda Tan
>         Attachments: YARN-7522.YARN-6592.002.patch, YARN-7522.YARN-6592.003.patch, YARN-7522.YARN-6592.wip-001.patch
> This is different from YARN-6596, YARN-6596 is targeted to add constraint manager to
store intra/inter application placement constraints. This JIRA is targeted to support storing
maps between container-tags/applications and nodes. This will be required by affinity/anti-affinity
implementation and cardinality.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message