Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: yarn-issues@hadoop.apache.org
Date: Fri, 10 Oct 2014 00:40:35 +0000 (UTC)
From: "Eric Payne (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.12714119.1400048094000.233465.1412901635806@Atlassian.JIRA>
In-Reply-To: <JIRA.12714119.1400048094000@Atlassian.JIRA>
References: <JIRA.12714119.1400048094000@Atlassian.JIRA>
 <JIRA.12714119.1400048094445@arcas>
Subject: [jira] [Commented] (YARN-2056) Disable preemption at Queue level
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/YARN-2056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14166073#comment-14166073 ] 

Eric Payne commented on YARN-2056:
----------------------------------

Thank you very much for your suggestions and review comments, [~leftnoteasy].

I am still working through your suggestion for a new algorithm.

In the meantime, I have analyzed the use case you provided, and I'm pretty sure the current patch does what is expected.

{code}
total = 100
qA: used = 10, guaranteed = 10, pending = 100
qB: used = 25, guaranteed = 10, pending = 100 (non-preemptable)
qC: used = 0, guaranteed = 80, pending = 0

1. Prior to the first round, qB is removed, from qAlloc and those resources are removed from unassigned.
    unassigned = 75
2. During the first round, qA is normalized to 0.11, and so is offered 8, and takes 8.
    qA.idealassigned = 8
    qC is removed from qAlloc
    unassigned = 67
3. During the second round, qA is normalized to 1.0 and offered 67. However, since offer() is also considering qB, offer() determines that qA needs 17 to get to the same level of over-capacity as qA, so offer() only selects 17 of the 67. Since qA didn't take the whole offer and also qA's requests are not satisfied, qA was not removed from qAlloc.
    qA.idealAssigned = 25
    unassigned = 50
    wQdone = 17
4. Since qA is now the same amount of over-capacity as qB, qB is added back.
5. In the third round, both qA and qB are normalized at 0.5 and are both offered 25, which completes the assignments:
    qA.idealAssigned = 50
    qB.idealAssigned = 50
{code}

> Disable preemption at Queue level
> ---------------------------------
>
>                 Key: YARN-2056
>                 URL: https://issues.apache.org/jira/browse/YARN-2056
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager
>    Affects Versions: 2.4.0
>            Reporter: Mayank Bansal
>            Assignee: Eric Payne
>         Attachments: YARN-2056.201408202039.txt, YARN-2056.201408260128.txt, YARN-2056.201408310117.txt, YARN-2056.201409022208.txt, YARN-2056.201409181916.txt, YARN-2056.201409210049.txt, YARN-2056.201409232329.txt, YARN-2056.201409242210.txt
>
>
> We need to be able to disable preemption at individual queue level


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)