Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: yarn-issues@hadoop.apache.org
Date: Thu, 10 Mar 2016 07:58:40 +0000 (UTC)
From: "Jian He (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.12861430.1441238461000.56103.1457596720914@Atlassian.JIRA>
In-Reply-To: <JIRA.12861430.1441238461000@Atlassian.JIRA>
References: <JIRA.12861430.1441238461000@Atlassian.JIRA>
 <JIRA.12861430.1441238461174@arcas>
Subject: [jira] [Commented] (YARN-4108) CapacityScheduler: Improve
 preemption to preempt only those containers that would satisfy the incoming
 request
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable


    [ https://issues.apache.org/jira/browse/YARN-4108?page=3Dcom.atlassian.=
jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D15188=
878#comment-15188878 ]=20

Jian He commented on YARN-4108:
-------------------------------

For the new logic in PCPP,  every round it=E2=80=99s looping all  previousl=
y calculated killable containers and check whether it=E2=80=99s finished, i=
f not, deducting the previously killable containers resource and then calcu=
lating the new extra set of containers to kill. I think in a simpler way, w=
e can re-generate the whole new set of containers(as it already does) every=
 round . Because in other scenarios, previously marked killable containers =
will may become invalid  too, then we need to repeat similar logic. That is=
:
- PCPP does not need to track the killableConainers.
- once PCPP calculated the containers to kill, replace that synchronously i=
n PreemptionManager.
- when looping the queues, as done in the patch, getKillableResource and ge=
tKillableContainers accordingly from PreemptionManager.


> CapacityScheduler: Improve preemption to preempt only those containers th=
at would satisfy the incoming request
> -------------------------------------------------------------------------=
-------------------------------------
>
>                 Key: YARN-4108
>                 URL: https://issues.apache.org/jira/browse/YARN-4108
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: capacity scheduler
>            Reporter: Wangda Tan
>            Assignee: Wangda Tan
>         Attachments: YARN-4108-design-doc-V3.pdf, YARN-4108-design-doc-v1=
.pdf, YARN-4108-design-doc-v2.pdf, YARN-4108.1.patch, YARN-4108.2.patch, YA=
RN-4108.3.patch, YARN-4108.4.patch, YARN-4108.5.patch, YARN-4108.6.patch, Y=
ARN-4108.poc.1.patch, YARN-4108.poc.2-WIP.patch, YARN-4108.poc.3-WIP.patch,=
 YARN-4108.poc.4-WIP.patch
>
>
> This is sibling JIRA for YARN-2154. We should make sure container preempt=
ion is more effective.
> *Requirements:*:
> 1) Can handle case of user-limit preemption
> 2) Can handle case of resource placement requirements, such as: hard-loca=
lity (I only want to use rack-1) / node-constraints (YARN-3409) / black-lis=
t (I don't want to use rack1 and host\[1-3\])
> 3) Can handle preemption within a queue: cross user preemption (YARN-2113=
), cross applicaiton preemption (such as priority-based (YARN-1963) / fairn=
ess-based (YARN-3319)).


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)