hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun Suresh (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6599) Support rich placement constraints in scheduler
Date Sat, 06 Jan 2018 07:58:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-6599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16314436#comment-16314436

Arun Suresh commented on YARN-6599:

bq. I can rename canSatisfySingleConstraint to canSatisfyConstraints, but it seems that we
still need a separate method. In this patch, we need to pass in PlacementConstraint directly.
Are you fine with this?
Makes Sense

Can we split the node-partition target expression handling aspect into a separate JIRA maybe
I would prefer not, there're only a few logics related to partition handling. Splitting this
logic cannot help reduce the size of the patch.
[~leftnoteasy], apologize, but I think we really should split this part. I see a lot of if
then also code related to node partitions peppered everywhere - which makes it a bit hard
to follow the code. Lets start with a target expression that can handle node/rack scope and
then build on that. [~kkaranasos], thoughts ?

bq. Also, I'm not sure what is the SELF as target type means
It just means source tag == target expression tag.

bq. I'm open to any feedbacks on this. I think this is documented in our design doc attached
to YARN-6592. Check chapter "Applying constraints during scheduling". 
I understand - but my issue with with regard to implementation and from a general usability
As I mentioned in the previous comment. There is is this issue of application priority - If
app1 has a lower priority than app2 but is scheduled first and app2 has issues scheduling
requests that depend on app1, allocation might stall. I think more thought should be put into

> Support rich placement constraints in scheduler
> -----------------------------------------------
>                 Key: YARN-6599
>                 URL: https://issues.apache.org/jira/browse/YARN-6599
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Wangda Tan
>            Assignee: Wangda Tan
>         Attachments: YARN-6599-YARN-6592.003.patch, YARN-6599-YARN-6592.004.patch, YARN-6599-YARN-6592.005.patch,
YARN-6599-YARN-6592.006.patch, YARN-6599-YARN-6592.007.patch, YARN-6599-YARN-6592.008.patch,
YARN-6599-YARN-6592.wip.002.patch, YARN-6599.poc.001.patch

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message