hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3275) Preemption happening on non-preemptable queues
Date Fri, 27 Feb 2015 23:25:05 GMT

    [ https://issues.apache.org/jira/browse/YARN-3275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14341014#comment-14341014

Wangda Tan commented on YARN-3275:

In addition, parent queue can go beyond max capacity as described in YARN-3243 no matter if
cluster resource changed or not. But child queue can only go beyond max capacity when cluster
resource reduced.

Eric, could you verify is this any of cases I mentioned above?
bq. No, I don't think going over the absolute max capacity should happen, in one case, we
saw a small queue in a very busy and large cluster go over by 5%

I feel maybe we need preempt container in your described case https://issues.apache.org/jira/browse/YARN-3275?focusedCommentId=14340393&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14340393,
capacity and disable-preemption are just two options, we haven't defined "disable-preemption"
is more important than "max-capacity". IMO, if we should do this JIRA or not is still discussable.


> Preemption happening on non-preemptable queues
> ----------------------------------------------
>                 Key: YARN-3275
>                 URL: https://issues.apache.org/jira/browse/YARN-3275
>             Project: Hadoop YARN
>          Issue Type: Bug
>    Affects Versions: 2.7.0
>            Reporter: Eric Payne
>            Assignee: Eric Payne
>         Attachments: YARN-3275.v1.txt
> YARN-2056 introduced the ability to turn preemption on and off at the queue level. In
cases where a queue goes over its absolute max capacity (YARN-3243, for example), containers
can be preempted from that queue, even though the queue is marked as non-preemptable.
> We are using this feature in large, busy clusters and seeing this behavior.

This message was sent by Atlassian JIRA

View raw message