hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6599) Support rich placement constraints in scheduler
Date Sat, 06 Jan 2018 00:32:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-6599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16314218#comment-16314218

Wangda Tan commented on YARN-6599:

Thanks [~asuresh] for reviewing this. 

bq. I think you should keep the name of the main entry point method as canSatisfyConstraints.
I can rename canSatisfySingleConstraint to canSatisfyConstraints, but it seems that we still
need a separate method. In this patch, we need to pass in PlacementConstraint directly. Are
you fine with this?

bq. Shouldnt you have to do the checkMinCardinality in all cases ?
No, if minCardinality = 0, we don't need to check it at all.

bq. Can we split the node-partition target expression handling aspect into a separate JIRA
maybe ?
I would prefer not, there're only a few logics related to partition handling. Splitting this
logic cannot help reduce the size of the patch. 

bq. To check if the given constraint is a SingleConstraint, you should use the transformer
first before doing a type cast. 
This part I do not quite understand, could you explain a bit? And another question is I'm
using following logic to copy SchedulingRequest, is it correct?
396     this.schedulingRequest = new SchedulingRequestPBImpl(
397         ((SchedulingRequestPBImpl) newSchedulingRequest).getProto());

bq. Can we target this JIRA for just intra-app anti-affinity. 
I'm open about this, could you respond to my comment below?

bq. I am also not really sold on the application-id/allocation-tag format yet.
I'm open to any feedbacks on this. I think this is documented in our design doc attached to
YARN-6592. Check chapter "Applying constraints during scheduling". 
We have to include anti-affinity to a specific app in this patch to support intra-app anti-affinity.
This is because by default allocation-tag specified inside PlacementConstraint should take
account of tags from all apps. Are you fine with anti-affinity to its own application by using
the syntax I proposed in the patch? (The only different between my implementation and design
doc is my impl includes a prefix). 
Also, I'm not sure what is the SELF as target type means, does it mean the same app/priority+allocationId/SchedulingRequest?
It is not part of design doc, I'm not sure what's the reason to include it.

> Support rich placement constraints in scheduler
> -----------------------------------------------
>                 Key: YARN-6599
>                 URL: https://issues.apache.org/jira/browse/YARN-6599
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Wangda Tan
>            Assignee: Wangda Tan
>         Attachments: YARN-6599-YARN-6592.003.patch, YARN-6599-YARN-6592.004.patch, YARN-6599-YARN-6592.005.patch,
YARN-6599-YARN-6592.006.patch, YARN-6599-YARN-6592.007.patch, YARN-6599-YARN-6592.008.patch,
YARN-6599-YARN-6592.wip.002.patch, YARN-6599.poc.001.patch

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message