hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6593) [API] Introduce Placement Constraint object
Date Tue, 23 May 2017 20:22:04 GMT

    [ https://issues.apache.org/jira/browse/YARN-6593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16021765#comment-16021765
] 

Wangda Tan commented on YARN-6593:
----------------------------------

[~kkaranasos], 

bq. That's exactly the advantage of having a single simple constraint. When it is a target
constraint, min cardinality is 0 and max cardinality is infinite. When it is a cardinality
constraint, target is yourself. This way the scheduler has to deal with a single placement
constraint. No need for checking constraint-types at any point.
This is what I want to avoid, implicitly define internal fields makes harder for scheduler
to handle and user to understand.

Pseudo code of you were describing is:
{code}
SimplePlacementConstraint c = ...;

if (c.minCardinality == 0 && c.maxCardinality == inf) {
  // this is a target constraint
  TargetExp t = c.getTargetExpression ...
} else if (c.target == null /* or self */ ) {
  int minCardinality = c.getMinCardinality();
  int maxCardinality = c.getMaxCardinality();
} else {
  // throw exception, not allow to set all 3 fields
}

// If we need get TargetConstraint / Cardinality in other places, we have to copy above logics
everywhere.
{code}

Instead, what we can do is
{code}
SimplePalcementConstraint c = ...;
if (c.getType() == TargetConstraint) {
  TargetConstraint tc = (TargetConstraint) c;
} else if (c.getType() == CardinalityConstraint) {
  CardinalityConstraint cc = (CardinalityConstraint) c;
}

// tc and cc can be saved, we don't have to do the cast every time ..
{code}

bq. We keep it simple for the applications to express constraints through the utility methods,
and we keep it simple for the scheduler to deal with a single constraint type.
But it cannot let applications to access constraints added easily.

bq. Moreover, if we have multiple classes for simple constraints, when building the objects
from the protobufs in the scheduler side, we will have to cast to each constraint class. And
if we decide to change those classes in the future (to expose a different way for users to
write constraints), we will have to change the way the scheduler deals with constraints, which
should really be avoided.

This is end user API, I don't think we should rapidly change this. YARN applications (like
native services) will consume this once it is available. That is why I want to keep it stable
and easy to use from the beginning.

> [API] Introduce Placement Constraint object
> -------------------------------------------
>
>                 Key: YARN-6593
>                 URL: https://issues.apache.org/jira/browse/YARN-6593
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Konstantinos Karanasos
>            Assignee: Konstantinos Karanasos
>         Attachments: YARN-6593.001.patch, YARN-6593.002.patch
>
>
> This JIRA introduces an object for defining placement constraints.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message