hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantinos Karanasos (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-7822) Constraint satisfaction checker support for composite OR and AND constraints
Date Mon, 29 Jan 2018 19:46:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-7822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16343897#comment-16343897
] 

Konstantinos Karanasos commented on YARN-7822:
----------------------------------------------

Patch looks good, thanks [~cheersyang].

I have only one ask, which I don't think is a big change. Can we support DNF? This is just
ORs of ANDs. This means that we can allow the following:
 * AND constraints with no nesting (your patch does that already);
 * OR constraints, each being either a simple constraint or an AND constraint with no nesting.
So essentially, you only have to allow this additional case in your patch.

If we do this, then we can transform any compound constraint with any nesting level to an
equivalent DNF form (we can tackle this in a separate JIRA), and we will be done with the
canSatisfyConstraint for all possible constraints.

 

Regarding the max cardinality, you are right it is a bit confusing in the design doc, but
it is not a code implementation artifact.

The idea is that the constraint has to be satisfied the moment of scheduling:
 * When the source and target tags are different, this happens to be the same both before
and after the placement. Assume a constraint that hb-m should be placed at a node with at
most 2 hb-rs. Say node n1 has before scheduling 2 hb-rs, so we can place hb-m there. After
placement, it will still have 2 hb-rs.
 * Now when the source and target tags are the same, it is more complicated. Assume a constraint
that hb-rs should be placed at a node with at most 2 hb-rs. Say node n1 has again 2 hb-rs
before scheduling. After scheduling, it will have 3.

So to unify the two cases above, we say that the constraints should be satisfied before the
placement of the new container happens. As [~asuresh] mentioned, at some point we tried to
create a special case for when source==target, but it the semantics were not straightforward
(e.g., if a user was saying max cardinality 2, they were expecting 2 after placement; when
a user says max cardinality 0, they think anti-affinity, but it actually has to be 1 if we
consider cardinality after placement). To unify things, we opted for cardinality check before
placement in all cases.

 

> Constraint satisfaction checker support for composite OR and AND constraints
> ----------------------------------------------------------------------------
>
>                 Key: YARN-7822
>                 URL: https://issues.apache.org/jira/browse/YARN-7822
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Arun Suresh
>            Assignee: Weiwei Yang
>            Priority: Major
>         Attachments: YARN-7822-YARN-6592.001.patch, YARN-7822-YARN-6592.002.patch, YARN-7822-YARN-6592.003.patch,
YARN-7822-YARN-6592.004.patch
>
>
> JIRA to track changes to {{PlacementConstraintsUtil#canSatisfyConstraints}} handle OR
and AND Composite constaints



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message