hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantinos Karanasos (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-8112) Fix min cardinality check for same source and target tags in intra-app constraints
Date Tue, 03 Apr 2018 15:45:00 GMT

     [ https://issues.apache.org/jira/browse/YARN-8112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Konstantinos Karanasos updated YARN-8112:
-----------------------------------------
    Description: 
The min cardinality constraint (min cardinality = _k_) ensures that a container is placed
at a node that has already k occurrences of the target tag. For example, a constraint _zk=3,CARDINALITY,NODE,hb,2,10_
will place each of the three zk containers on a node with at least 2 hb instances (and no
more than 10 for the max cardinality).

Affinity constraints is a special case of this where min cardinality is 1.

Currently we do not support min cardinality when the source and the target of the constraint
are the same in an intra-app constraint.

Therefore, zk=3,CARDINALITY,NODE,zk,2,10 is not supported, and neither is zk=3,IN,NODE,zk.

This Jira will address this problem by placing the first k containers on the same node (or
any other specified scope, e.g., rack), so that min cardinality can be met when placing the
subsequent containers with the same tag.

  was:
The min cardinality constraint is 

 

Currently following PC doesn't work 
{code:java}
-placement_spec zk=3,IN,NODE,zk
-placement_spec zk=1,IN,NODE,zk
{code}
Expectation: place 3/1 zk containers in any node of the cluster.

This is because when we do cardinality check, PC cannot be satisfied as there is no "zk" tag
placed on any node. But this is a very common user-case, hence we need to support.


> Fix min cardinality check for same source and target tags in intra-app constraints
> ----------------------------------------------------------------------------------
>
>                 Key: YARN-8112
>                 URL: https://issues.apache.org/jira/browse/YARN-8112
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager
>            Reporter: Weiwei Yang
>            Priority: Major
>
> The min cardinality constraint (min cardinality = _k_) ensures that a container is placed
at a node that has already k occurrences of the target tag. For example, a constraint _zk=3,CARDINALITY,NODE,hb,2,10_
will place each of the three zk containers on a node with at least 2 hb instances (and no
more than 10 for the max cardinality).
> Affinity constraints is a special case of this where min cardinality is 1.
> Currently we do not support min cardinality when the source and the target of the constraint
are the same in an intra-app constraint.
> Therefore, zk=3,CARDINALITY,NODE,zk,2,10 is not supported, and neither is zk=3,IN,NODE,zk.
> This Jira will address this problem by placing the first k containers on the same node
(or any other specified scope, e.g., rack), so that min cardinality can be met when placing
the subsequent containers with the same tag.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message