hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Payne (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2056) Disable preemption at Queue level
Date Fri, 10 Oct 2014 00:40:35 GMT

    [ https://issues.apache.org/jira/browse/YARN-2056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14166073#comment-14166073
] 

Eric Payne commented on YARN-2056:
----------------------------------

Thank you very much for your suggestions and review comments, [~leftnoteasy].

I am still working through your suggestion for a new algorithm.

In the meantime, I have analyzed the use case you provided, and I'm pretty sure the current
patch does what is expected.

{code}
total = 100
qA: used = 10, guaranteed = 10, pending = 100
qB: used = 25, guaranteed = 10, pending = 100 (non-preemptable)
qC: used = 0, guaranteed = 80, pending = 0

1. Prior to the first round, qB is removed, from qAlloc and those resources are removed from
unassigned.
    unassigned = 75
2. During the first round, qA is normalized to 0.11, and so is offered 8, and takes 8.
    qA.idealassigned = 8
    qC is removed from qAlloc
    unassigned = 67
3. During the second round, qA is normalized to 1.0 and offered 67. However, since offer()
is also considering qB, offer() determines that qA needs 17 to get to the same level of over-capacity
as qA, so offer() only selects 17 of the 67. Since qA didn't take the whole offer and also
qA's requests are not satisfied, qA was not removed from qAlloc.
    qA.idealAssigned = 25
    unassigned = 50
    wQdone = 17
4. Since qA is now the same amount of over-capacity as qB, qB is added back.
5. In the third round, both qA and qB are normalized at 0.5 and are both offered 25, which
completes the assignments:
    qA.idealAssigned = 50
    qB.idealAssigned = 50
{code}

> Disable preemption at Queue level
> ---------------------------------
>
>                 Key: YARN-2056
>                 URL: https://issues.apache.org/jira/browse/YARN-2056
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager
>    Affects Versions: 2.4.0
>            Reporter: Mayank Bansal
>            Assignee: Eric Payne
>         Attachments: YARN-2056.201408202039.txt, YARN-2056.201408260128.txt, YARN-2056.201408310117.txt,
YARN-2056.201409022208.txt, YARN-2056.201409181916.txt, YARN-2056.201409210049.txt, YARN-2056.201409232329.txt,
YARN-2056.201409242210.txt
>
>
> We need to be able to disable preemption at individual queue level



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message