Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: yarn-issues@hadoop.apache.org
Date: Thu, 14 Apr 2016 21:07:25 +0000 (UTC)
From: "Wangda Tan (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.12957920.1460419286000.234730.1460668045509@Atlassian.JIRA>
In-Reply-To: <JIRA.12957920.1460419286000@Atlassian.JIRA>
References: <JIRA.12957920.1460419286000@Atlassian.JIRA>
 <JIRA.12957920.1460419286720@arcas>
Subject: [jira] [Commented] (YARN-4945) [Umbrella] Capacity Scheduler
 Preemption Within a queue
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/YARN-4945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15241922#comment-15241922 ] 

Wangda Tan commented on YARN-4945:
----------------------------------

[~eepayne],

Above is one of the case that needs intra-queue preemption kick in.

In my mind, intra-queue preemption could happen when the queue is not able to get new resources (1), and any of apps in the queue is under its ideal allocation. (2)

(1) The queue is not able to get new resources:
Not only queue has 100% resource usage and cluster is full. It is possible that because of queue's max capacity setting, queue cannot get more resource when cluster has sufficient idle resource .
And also, when some queue doesn't allow preemption, queue cannot get more resource when the queue below its guaranteed capacity. 

(2) Any of apps under its ideal allocation:
Not only for user-limit example, we may need to consider a general solution for different queue policies. For example: for Fifo+priority policy, highest application can take all capacities; for fair policy, app's ideal allocation is computed fair share.
The only difference between different intra-queue preemption goal (children tasks of this umbrella JIRA) is, we need to compute applications' ideal allocations in different way. Remain part should be same.

Thoughts?


> [Umbrella] Capacity Scheduler Preemption Within a queue
> -------------------------------------------------------
>
>                 Key: YARN-4945
>                 URL: https://issues.apache.org/jira/browse/YARN-4945
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Wangda Tan
>
> This is umbrella ticket to track efforts of preemption within a queue to support features like:
> YARN-2009. YARN-2113. YARN-4781.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)