hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Payne (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3275) Preemption happening on non-preemptable queues
Date Fri, 27 Feb 2015 17:18:05 GMT

    [ https://issues.apache.org/jira/browse/YARN-3275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14340393#comment-14340393
] 

Eric Payne commented on YARN-3275:
----------------------------------

This situation can happen under the following conditions:

- All of the resources in the cluster are being used
- {{QueueA}} is preemptable and over its absolute capacity (AKA guaranteed capacity)
- {{QueueB}} is not preemptable, over its absolute capacity, also over its absolute max capacity
(which can happen), and asking for more resources

In the above scenario, {{ProportionalCapacityPreemptionPolicy}} will subtract {{QueuB}}'s
ideal assigned value from its absolute max capacity value and get a negative number, which
will adjust it's ideally assigned resources downwards by that amount, which will result in
that amount getting preempted.

Regardless of the reason, if a queue is marked as unpreemptable, resources should never be
preempted from that queue.


> Preemption happening on non-preemptable queues
> ----------------------------------------------
>
>                 Key: YARN-3275
>                 URL: https://issues.apache.org/jira/browse/YARN-3275
>             Project: Hadoop YARN
>          Issue Type: Bug
>    Affects Versions: 2.7.0
>            Reporter: Eric Payne
>            Assignee: Eric Payne
>
> YARN-2056 introduced the ability to turn preemption on and off at the queue level. In
cases where a queue goes over its absolute max capacity (YARN:3243, for example), containers
can be preempted from that queue, even though the queue is marked as non-preemptable.
> We are using this feature in large, busy clusters and seeing this behavior.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message