hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Payne (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2056) Disable preemption at Queue level
Date Wed, 12 Nov 2014 14:30:34 GMT

    [ https://issues.apache.org/jira/browse/YARN-2056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14208076#comment-14208076

Eric Payne commented on YARN-2056:

[~leftnoteasy], thank you very much for taking the time to review this patch.
However, you cannot say, in this case, b cannot preempt resource from a. Because, When a container
preempted, the resource is available for everyone to use. Like the resource freed after preempt
a container from a2 that doesn't mean the resource is dedicated to a1 to use only.
Good point. I see what you mean.
If you agree, I would like to keep the inheritance behavior. That is, be able set the disable
preemption property at the parent level and have it trickle down, but only have the disable
preemption property take effect at the leaf level. The property could be overridden at the
leaf level.

There were actually 2 aspects of the latest patch:
# Set {{disable_preemption}} property at a parent level and have children inherit from that
unless  overridden.
#* Use Case: A cluster with many subqueues where most will have queue preemption disabled
with only one enabled.
#* Disable queue preemption at the root, and then enable it for the one leaf queue that will
be preemptable.
#* With this inheritance behavior, this requires a lot less {{disable_preemption}} settings.
# Enforce the {{disable_preemption}} property for parent queues.

If it's okay with you, I would like to keep number 1. If I remove number 2, but keep number
1, the behavior would be that the {{disable_preemption}} settings could be done in the parent,
but they would only affect the leaf queues.

bq. 1. getUnderservedQueues the name is a little confusing me, it should be getMostUnderServedQueues
Yes, I can do that.

bq.  2. I would suggest to create a method in CapacitySchedulerConfiguration.getPreemptionEnabled(queue)
I would like to open a new Jira that adds to the scheduler UI and logs the queue preemption
state. Does it make sense to add this API as part of that Jira?

bq.  3. In getUnderservedQueues, you can simple use tqComparator.compare(q0, q1) instead of
calculate idealPctGuaranteed
Okay. Sounds good.

bq. 4. it's better to rename idealPctGuaranteed -> caculate(or get)IdealPctGuaranteed.

> Disable preemption at Queue level
> ---------------------------------
>                 Key: YARN-2056
>                 URL: https://issues.apache.org/jira/browse/YARN-2056
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager
>    Affects Versions: 2.4.0
>            Reporter: Mayank Bansal
>            Assignee: Eric Payne
>         Attachments: YARN-2056.201408202039.txt, YARN-2056.201408260128.txt, YARN-2056.201408310117.txt,
YARN-2056.201409022208.txt, YARN-2056.201409181916.txt, YARN-2056.201409210049.txt, YARN-2056.201409232329.txt,
YARN-2056.201409242210.txt, YARN-2056.201410132225.txt, YARN-2056.201410141330.txt, YARN-2056.201410232244.txt,
YARN-2056.201410311746.txt, YARN-2056.201411041635.txt, YARN-2056.201411072153.txt
> We need to be able to disable preemption at individual queue level

This message was sent by Atlassian JIRA

View raw message