hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tao Yang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-8917) Absolute (maximum) capacity of level3+ queues is wrongly calculated for absolute resource
Date Sat, 20 Oct 2018 01:21:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-8917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16657637#comment-16657637

Tao Yang commented on YARN-8917:

Thanks [~sunilg], [~leftnoteasy] for the review.
cud u pls write a test case to prove the calculation is wrong in level+3 queues. That will
help to really see the problem in tests and easy to understand.
This patch tests the calculation for absolute capacity and absolute max capacity for queues
of all levels in TestAbsoluteResourceConfiguration#testEffectiveResourceAfterIncreasingClusterResource,
this will help to see problem on level 3+ queues if running without the fix. 
I will update v2 patch to add assert annotations in UT. 

> Absolute (maximum) capacity of level3+ queues is wrongly calculated for absolute resource
> -----------------------------------------------------------------------------------------
>                 Key: YARN-8917
>                 URL: https://issues.apache.org/jira/browse/YARN-8917
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: capacityscheduler
>    Affects Versions: 3.2.1
>            Reporter: Tao Yang
>            Assignee: Tao Yang
>            Priority: Major
>         Attachments: YARN-8917.001.patch, YARN-8917.002.patch
> Absolute capacity should be equal to multiply capacity by parent-queue's absolute-capacity,
> but currently it's calculated as dividing capacity by parent-queue's absolute-capacity.
> Calculation for absolute-maximum-capacity has the same problem.
> For example: 
> root.a   capacity=0.4   maximum-capacity=0.8
> root.a.a1   capacity=0.5  maximum-capacity=0.6
> Absolute capacity of root.a.a1 should be 0.2 but is wrongly calculated as 1.25
> Absolute maximum capacity of root.a.a1 should be 0.48 but is wrongly calculated as 0.75
> Moreover:
> {{childQueue.getQueueCapacities().getCapacity()}}  should be changed to {{childQueue.getQueueCapacities().getCapacity(label)}}
to avoid getting wrong capacity from default partition when calculating for a non-default

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message