hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Feng Yuan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6021) When your allocated minShare of all queue`s added up exceed cluster capacity you can get some queue for 0 fairshare
Date Tue, 27 Dec 2016 04:13:58 GMT

    [ https://issues.apache.org/jira/browse/YARN-6021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15779562#comment-15779562

Feng Yuan commented on YARN-6021:

[~kasha],assumpt this scenario:
capacity is 10,queue a,b,c minshare is 3,3,4.Now it is allright.But when you add queue d,you
need reconfig all of four queues minshare to make sure four queue work as expected(2,2,2,3)--on
this occasion you need 
coordinate four queues minshare(you need config 4 times),and this thing is infinite,when you
add queue e,f,g...
So as for me,it is more suitable to config 1 times.
add queue e,config once such as 4(because you dont need to care the upper limit)...
And one thing is more important——we cant ensure all queues will be active,so if according
to what you said,there is
only a part of queues active,and all these queue`s minshare added up is less than capacity
i think this maybe is waste.(I am not very confirm.)

> When your allocated minShare of all queue`s added up exceed cluster capacity you can
get some queue for 0 fairshare
> -------------------------------------------------------------------------------------------------------------------
>                 Key: YARN-6021
>                 URL: https://issues.apache.org/jira/browse/YARN-6021
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: fairscheduler
>    Affects Versions: 2.4.0
>            Reporter: Feng Yuan
>            Assignee: Feng Yuan
>            Priority: Critical
> In fair-scheduler.xml,If you config the minshare add up exceed parentQueue`s fairshare,for
root`s childs,fairshare is cluster capacity.
> You will found your R value look like below when compute childs fairshares:
> 1.0 
> 0.5 
> 0.25 
> 0.125 
> 0.0625 
> 0.03125 
> 0.015625 
> 0.0078125 
> 0.00390625
> I find this is due to:
> double rMax = 1.0;
>     while (resourceUsedWithWeightToResourceRatio(rMax, schedulables, type)
>         < totalResource) {
>       rMax *= 2.0;
>   }
> because resourceUsedWithWeightToResourceRatio will add minShare together.
> As i think is really should we bring in minShare when compute fairshare?
> My advice is we just consider weight is enough,and minshare's guarantee
> will get fulfill when assginContainer!
> Hope suggestion!

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message