hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2285) Preemption can cause capacity scheduler to show 5,000% queue capacity.
Date Wed, 16 Jul 2014 17:04:06 GMT

    [ https://issues.apache.org/jira/browse/YARN-2285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14063725#comment-14063725
] 

Wangda Tan commented on YARN-2285:
----------------------------------

Thanks comments from Vinod and Sunil,

bq. From the look of it, it sounds like this isn't tied to preemption. It looks like this
was a bug that exists even when preemption is not enabled. Can we validate that?
I'll validate this tomorrow

The usage of root queue above 100% is caused by reserved container, currently the UI shows
queue allocated+reserved, we may need change that for user easier understand what happened.

> Preemption can cause capacity scheduler to show 5,000% queue capacity.
> ----------------------------------------------------------------------
>
>                 Key: YARN-2285
>                 URL: https://issues.apache.org/jira/browse/YARN-2285
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: capacityscheduler
>    Affects Versions: 2.5.0
>         Environment: Turn on CS Preemption.
>            Reporter: Tassapol Athiapinya
>            Assignee: Wangda Tan
>            Priority: Minor
>         Attachments: preemption_5000_percent.png
>
>
> I configure queue A, B to have 1%, 99% capacity respectively. There is no max capacity
for each queue. Set high user limit factor.
> Submit app 1 to queue A. AM container takes 50% of cluster memory. Task containers take
another 50%. Submit app 2 to queue B. Preempt task containers of app 1 out. Turns out capacity
of queue B increases to 99% but queue A has 5000% used.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message