Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: yarn-issues@hadoop.apache.org
Date: Wed, 16 Dec 2015 01:06:46 +0000 (UTC)
From: "Wangda Tan (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.12861430.1441238461000.33722.1450228006822@Atlassian.JIRA>
In-Reply-To: <JIRA.12861430.1441238461000@Atlassian.JIRA>
References: <JIRA.12861430.1441238461000@Atlassian.JIRA>
 <JIRA.12861430.1441238461174@arcas>
Subject: [jira] [Updated] (YARN-4108) CapacityScheduler: Improve preemption
 to preempt only those containers that would satisfy the incoming request
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/YARN-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Wangda Tan updated YARN-4108:
-----------------------------
    Attachment: YARN-4108-design-doc-v2.pdf

Had an offline discussion with [~curino]/[~chris.douglas]. Updated design doc (v2), highlight of changes:
- Added an example for people to better understand how lazy preemption works
- Added more explanations about architecture design / implementation details
- Added two items for discussion
1) Is it possible to combine batch preemption and surgical preemption.
2) Is it possible to kill preempted container processes at NM only when new container is ready to launch.

Thanks [~curino]/[~chris.douglas] for very insightful thoughts and suggestions!

> CapacityScheduler: Improve preemption to preempt only those containers that would satisfy the incoming request
> --------------------------------------------------------------------------------------------------------------
>
>                 Key: YARN-4108
>                 URL: https://issues.apache.org/jira/browse/YARN-4108
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: capacity scheduler
>            Reporter: Wangda Tan
>            Assignee: Wangda Tan
>         Attachments: YARN-4108-design-doc-v1.pdf, YARN-4108-design-doc-v2.pdf, YARN-4108.poc.1.patch
>
>
> This is sibling JIRA for YARN-2154. We should make sure container preemption is more effective.
> *Requirements:*:
> 1) Can handle case of user-limit preemption
> 2) Can handle case of resource placement requirements, such as: hard-locality (I only want to use rack-1) / node-constraints (YARN-3409) / black-list (I don't want to use rack1 and host\[1-3\])
> 3) Can handle preemption within a queue: cross user preemption (YARN-2113), cross applicaiton preemption (such as priority-based (YARN-1963) / fairness-based (YARN-3319)).


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)