hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Hung (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5293) Disable preemption per queue per partition
Date Mon, 27 Jun 2016 19:30:52 GMT

    [ https://issues.apache.org/jira/browse/YARN-5293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15351650#comment-15351650

Jonathan Hung commented on YARN-5293:

Thanks for the comments, [~Naganarasimha] and [~sunilg]. My initial idea was to keep the preemption
disabled a queue level configuration. So we would have something like {{yarn.scheduler.capacity.root.queueQ.accessible-node-labels.labelA.disable_preemption}}
(say this is set to true). This would override the configuration {{yarn.scheduler.capacity.root.queueQ.disable_preemption}}
(say this is set to false) for labelA, so queueQ should not be preempted for labelA but will
be preempted for other labels. 

For the case where {{yarn.scheduler.capacity.root.queueQ.accessible-node-labels.labelA.disable_preemption}}
is not specified, it should inherit from its parent's per-queue-per-label configuration (in
this case, {{yarn.scheduler.capacity.root.accessible-node-labels.labelA.disable_preemption}}),
just as it does in the existing code for unlabeled disable_preemption.

Can't currently think of any use cases for disabling preemption for a label. If we wanted
to do this, we could have a configuration like {{yarn.scheduler.capacity.labelA.disable_preemption}}.
If neither {{yarn.scheduler.capacity.root.queueQ.accessible-node-labels.labelA.disable_preemption}}
or {{yarn.scheduler.capacity.root.accessible-node-labels.labelA.disable_preemption}} is specified,
it will inherit from {{yarn.scheduler.capacity.labelA.disable_preemption}}. 

One of the results of this is that {{yarn.scheduler.capacity.root.queueQ.disable_preemption}}
no longer has any effect on disabling preemption for queueQ in labeled partitions. So if a
user wanted to disable preemption for a queue (across all partitions), they would have to
specify this for each partition.

> Disable preemption per queue per partition
> ------------------------------------------
>                 Key: YARN-5293
>                 URL: https://issues.apache.org/jira/browse/YARN-5293
>             Project: Hadoop YARN
>          Issue Type: Improvement
>            Reporter: Jonathan Hung
> [YARN-2056|https://issues.apache.org/jira/browse/YARN-2056] added support for disabling
preemption at queue level. [YARN-2498|https://issues.apache.org/jira/browse/YARN-2498] added
support for per partition preemption. It would be useful to be able to disable preemption
per partition per queue, so that if a queue Q has access to two labels A and B, we can disable
preemption for Q in A but not in B.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message