hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantinos Karanasos (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-7778) Merging of constraints defined at different levels
Date Wed, 31 Jan 2018 19:59:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-7778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16347514#comment-16347514

Konstantinos Karanasos commented on YARN-7778:

Thanks for the patch, [~cheersyang]. A couple of comments:
 * Let's rename the getResourceConstraint() to something like getMultilevelConstraint? Also
the initial part of the comment "Retrieve the placement constraint of a given scheduling
request from the PCM point of view" could be made more like "Consider all levels of constraints
(resource request, app, cluster) and return a merged constraint". This is because we can also
return a merged constraint of app and cluster level, even if there are no resource request
 * I would also change the signature of the method to add directly a Constraint coming from
the resource request and a set of tags, rather than pass the SchedulingRequest object. This
way we make clear that the set of tags is separate from the specific constraint.
 * Do we need the "Remove all null or duplicate constraints"? I was thinking that this should
belong to a separate step that will do constraint simplification/minimization. If we keep
it, I would suggest to put it in a separate method in the PCM, so that we can use it elsewhere
too or extend it in the future easily.

> Merging of constraints defined at different levels
> --------------------------------------------------
>                 Key: YARN-7778
>                 URL: https://issues.apache.org/jira/browse/YARN-7778
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Konstantinos Karanasos
>            Assignee: Weiwei Yang
>            Priority: Major
>         Attachments: Merge Constraints Solution.pdf, YARN-7778-YARN-7812.001.patch, YARN-7778-YARN-7812.002.patch
> When we have multiple constraints defined for a given set of allocation tags at different
levels (i.e., at the cluster, the application or the scheduling request level), we need to
merge those constraints.
> Defining constraint levels as cluster > application > scheduling request, constraints
defined at lower levels should only be more restrictive than those of higher levels. Otherwise
the allocation should fail.
> For example, if there is an application level constraint that allows no more than 5 HBase
containers per rack, a scheduling request can further restrict that to 3 containers per rack
but not to 7 containers per rack.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message