hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karthik Kambatla (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5453) FairScheduler#update may skip update demand resource of child queue/app if current demand reached maxResource
Date Tue, 09 Aug 2016 15:08:22 GMT

    [ https://issues.apache.org/jira/browse/YARN-5453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15413687#comment-15413687
] 

Karthik Kambatla commented on YARN-5453:
----------------------------------------

Demand is limited to maxResources to enforce maxResources: If we don't limit the demand, demand
could be larger than maxResources. Even though the fairshare is limited by maxResources, the
scheduler allows allocations over fairshare and hence over maxResources. 

We should document this in the code itself for posterity. 

In the snippet posted in the description, why not change {{break}} to {{continue}} and move
{{demand = componentWiseMin(demand, maxRes);}} outside the for loop. 

> FairScheduler#update may skip update demand resource of child queue/app if current demand
reached maxResource
> -------------------------------------------------------------------------------------------------------------
>
>                 Key: YARN-5453
>                 URL: https://issues.apache.org/jira/browse/YARN-5453
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: sandflee
>            Assignee: sandflee
>         Attachments: YARN-5453.01.patch
>
>
> {code}
>       demand = Resources.createResource(0);
>       for (FSQueue childQueue : childQueues) {
>         childQueue.updateDemand();
>         Resource toAdd = childQueue.getDemand();
>         demand = Resources.add(demand, toAdd);
>         demand = Resources.componentwiseMin(demand, maxRes);
>         if (Resources.equals(demand, maxRes)) {
>           break;
>         }
>       }
> {code}
> if one singe queue's demand resource exceed maxRes,  the other queue's demand resource
will not update.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message