hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yufei Gu (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (YARN-5077) Fix FSLeafQueue#getFairShare() for queues with weight 0.0
Date Thu, 26 May 2016 17:47:13 GMT

    [ https://issues.apache.org/jira/browse/YARN-5077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301694#comment-15301694
] 

Yufei Gu edited comment on YARN-5077 at 5/26/16 5:47 PM:
---------------------------------------------------------

[~kasha], I have this concern as well. If a queue can get resources assigned even if the fairshare
is zero, why we have this bug. I first tested it with a unmanaged AM. It can get the resources
even its weight is zero. Then a managed AM cannot get the resources because its maxAMResource
always is zero, and maxAMResource is calculated by multiplying faireshare and maxAMShare.


Do you suggest we can change the algorithm of calculating maxAMResource to solve this problem?


was (Author: yufeigu):
[~kasha], That's my first reaction when you introduced how it works. If a queue can get resources
assigned even if the fairshare is zero, why we have this bug. I first tested it with a unmanaged
AM. It can get the resources even its weight is zero. Then a managed AM cannot get the resources
because its maxAMResource always is zero, and maxAMResource is calculated by multiplying faireshare
and maxAMShare. 

Do you suggest we can change the algorithm of calculating maxAMResource to solve this problem?

> Fix FSLeafQueue#getFairShare() for queues with weight 0.0
> ---------------------------------------------------------
>
>                 Key: YARN-5077
>                 URL: https://issues.apache.org/jira/browse/YARN-5077
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Yufei Gu
>            Assignee: Yufei Gu
>         Attachments: YARN-5077.001.patch, YARN-5077.002.patch, YARN-5077.003.patch, YARN-5077.004.patch,
YARN-5077.005.patch
>
>
> 1) When a queue's weight is set to 0.0, FSLeafQueue#getFairShare() returns <memory:0,
vCores:0> 
> 2) When a queue's weight is nonzero, FSLeafQueue#getFairShare() returns <memory:16384,
vCores:8>
> In case 1), that means no container ever gets allocated for an AM because from the viewpoint
of the RM, there is never any headroom to allocate a container on that queue.
> For example, we have a pool with the following weights: 
> - root.dev 0.0 
> - root.product 1.0
> The root.dev is a best effort pool and should only get resources if root.product is not
running. In our tests, with no jobs running under root.product, jobs started in root.dev queue
stay stuck in ACCEPT phase and never start.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message