hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4945) [Umbrella] Capacity Scheduler Preemption Within a queue
Date Thu, 14 Apr 2016 21:07:25 GMT

    [ https://issues.apache.org/jira/browse/YARN-4945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15241922#comment-15241922
] 

Wangda Tan commented on YARN-4945:
----------------------------------

[~eepayne],

Above is one of the case that needs intra-queue preemption kick in.

In my mind, intra-queue preemption could happen when the queue is not able to get new resources
(1), and any of apps in the queue is under its ideal allocation. (2)

(1) The queue is not able to get new resources:
Not only queue has 100% resource usage and cluster is full. It is possible that because of
queue's max capacity setting, queue cannot get more resource when cluster has sufficient idle
resource .
And also, when some queue doesn't allow preemption, queue cannot get more resource when the
queue below its guaranteed capacity. 

(2) Any of apps under its ideal allocation:
Not only for user-limit example, we may need to consider a general solution for different
queue policies. For example: for Fifo+priority policy, highest application can take all capacities;
for fair policy, app's ideal allocation is computed fair share.
The only difference between different intra-queue preemption goal (children tasks of this
umbrella JIRA) is, we need to compute applications' ideal allocations in different way. Remain
part should be same.

Thoughts?


> [Umbrella] Capacity Scheduler Preemption Within a queue
> -------------------------------------------------------
>
>                 Key: YARN-4945
>                 URL: https://issues.apache.org/jira/browse/YARN-4945
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Wangda Tan
>
> This is umbrella ticket to track efforts of preemption within a queue to support features
like:
> YARN-2009. YARN-2113. YARN-4781.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message