hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Payne (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-2056) Disable preemption at Queue level
Date Tue, 23 Sep 2014 23:55:35 GMT

     [ https://issues.apache.org/jira/browse/YARN-2056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Eric Payne updated YARN-2056:
-----------------------------
    Attachment: YARN-2056.201409232329.txt

[~leftnoteasy],

This patch now works in all cases, except IF queue preemption is disabled for a particular
queue, it has a different behavior than previously. That is:
{code}
root has A,B,C, total capacity = 90
A.guaranteed = 30, A.pending = 20, A.current = 40
B.guaranteed = 30, B.pending = 0, B.current = 50
C.guaranteed = 30, C.pending = 0, C.current = 0
{code}
In the above case, if all queues are preemptable, this patch works the same as it did before.
That is, A and B will both end up with 45.

However, with this patch
{code}
IF (A is not preemptable)
    AND (A is already over capacity)
    AND (all resources are used)
    AND (A is asking for more resources) {
        A will remain at 40 and B will remain at 50
}
{code}
I believe that there is a way to make the this patch maintain the old behavior, even if A
is not preemptable, but it would require something like the following algorithm:
{code}
In ProportionalCapacityPreemptionPolicy#computeFixpointAllocation:
FOR each queue {
    IF queue has untouchableExtra {
        queue.idalAssigned = queue.guaranteed + queue.untouchable
        unassigned -= queue.idealAssigned
        remove queue from qAlloc
        add queue to list of queues that were removed
    }
}

Assign the remaining unassigned resources, computing idealAssigned for the remaining queues,
with the following modification:
    IF (queues at this level go over their capacity)
        AND (they are over by the same percentage as the queue(s) that were removed) {
        put the removed queue(s) back into qAlloc and continue to compute idealAssigned
    }
{code}


> Disable preemption at Queue level
> ---------------------------------
>
>                 Key: YARN-2056
>                 URL: https://issues.apache.org/jira/browse/YARN-2056
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager
>    Affects Versions: 2.4.0
>            Reporter: Mayank Bansal
>            Assignee: Eric Payne
>         Attachments: YARN-2056.201408202039.txt, YARN-2056.201408260128.txt, YARN-2056.201408310117.txt,
YARN-2056.201409022208.txt, YARN-2056.201409181916.txt, YARN-2056.201409210049.txt, YARN-2056.201409232329.txt
>
>
> We need to be able to disable preemption at individual queue level



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message