hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4415) Scheduler Web Ui shows max capacity for the queue is 100% but when we submit application doesnt get assigned
Date Mon, 07 Dec 2015 21:46:11 GMT

    [ https://issues.apache.org/jira/browse/YARN-4415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15045800#comment-15045800
] 

Wangda Tan commented on YARN-4415:
----------------------------------

Thanks for looking at this issue, [~Naganarasimha]/[~xinxianyin].

I think QueueCapacitiesInfo should not assume maxCapacity will be > eps. We have normalizations
while setting values to QueueCapacities, so we should copy exactly same value from QueueCapacities
to QueueCapacitiesInfo (cap it between 0 and 1 is fine).

It's a valid use case that a queue has max capacity = 0, for example, reservation system (YARN-1051)
could dynamically adjust queue capacities.

bq. IMO we need to fetch the capacities of a partition for a given queue from its parent,
if capacities are not configured for it. And if its not configured to its parent then from
its parent's capacities. if the root itself doesnt have then its should be 0 as capacity and
100 as max capacity
I may not fully understand why we need to fetch parent queue's capacities while setting QueueCapacitiesInfo.
As I mentioned above, QueueCapacities should have everything considered and calculated at
QueueCapacities (including parent queue's capacities), correct?

Thoughts?

> Scheduler Web Ui shows max capacity for the queue is 100% but when we submit application
doesnt get assigned
> ------------------------------------------------------------------------------------------------------------
>
>                 Key: YARN-4415
>                 URL: https://issues.apache.org/jira/browse/YARN-4415
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: capacity scheduler, resourcemanager
>    Affects Versions: 2.7.2
>            Reporter: Naganarasimha G R
>            Assignee: Naganarasimha G R
>         Attachments: App info with diagnostics info.png, screenshot-1.png
>
>
> Steps to reproduce the issue :
> Scenario 1:
> # Configure a queue(default) with accessible node labels as *
> # create a exclusive partition *xxx* and map a NM to it
> # ensure no capacities are configured for default for label xxx
> # start an RM app with queue as default and label as xxx
> # application is stuck but scheduler ui shows 100% as max capacity for that queue
> Scenario 2:
> # create a nonexclusive partition *sharedPartition* and map a NM to it
> # ensure no capacities are configured for default queue
> # start an RM app with queue as *default* and label as *sharedPartition*
> # application is stuck but scheduler ui shows 100% as max capacity for that queue for
*sharedPartition*
> For both issues cause is the same default max capacity and abs max capacity is set to
Zero %



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message