hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Feng Yuan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6021) When your allocated minShare of all queue`s added up exceed cluster capacity you can get some queue for 0 fairshare
Date Sat, 24 Dec 2016 04:31:58 GMT

    [ https://issues.apache.org/jira/browse/YARN-6021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15774257#comment-15774257
] 

Feng Yuan commented on YARN-6021:
---------------------------------

Hi,[~kasha]:
          root
          / | \
         a  b  c
Assumpt,root fairshare(fs) is 10.
And minshare of childs:
a:6
b:5
c:0
Procedure is:
1.When compute fs in ComputeFairShares#computeSharesInternal:
2.In function resourceUsedWithWeightToResourceRatio,It will 
find a,b,c minshare add up greater than 10.
3.Then jump out "while (resourceUsedWithWeightToResourceRatio(rMax, schedulables, type)".
4.Go into for (int i = 0; i < COMPUTE_FAIR_SHARES_ITERATIONS; i++) {...}
5.When jump out 4th ,R value is a very wee num.
6.Then when go into setResourceValue(computeShare(sched, right, type), sched.getFairShare(),
type); 
7.Will find queue c fairshare is ZERO.--(int)Max(R*weight,minShare)

So in the beginning,My viewpoint is:
If a queue(c) is active,its fairshare could not be 0.

And i have some questions:
1.Need refer to minshare when compute fairshare?
2.Childs minshare added up exceeded parents fairshare——(as for root fairshare is cluster
capacity.),
Is this situation rightful? 
Should we check this rule in load configuration file? 
How could we control users behavior?

Hope any suggestions. Thanks!


> When your allocated minShare of all queue`s added up exceed cluster capacity you can
get some queue for 0 fairshare
> -------------------------------------------------------------------------------------------------------------------
>
>                 Key: YARN-6021
>                 URL: https://issues.apache.org/jira/browse/YARN-6021
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: fairscheduler
>    Affects Versions: 2.4.0
>            Reporter: Feng Yuan
>            Assignee: Feng Yuan
>            Priority: Critical
>
> In fair-scheduler.xml,If you config the minshare add up exceed parentQueue`s fairshare,for
root`s childs,fairshare is cluster capacity.
> You will found your R value look like below when compute childs fairshares:
> 1.0 
> 0.5 
> 0.25 
> 0.125 
> 0.0625 
> 0.03125 
> 0.015625 
> 0.0078125 
> 0.00390625
> I find this is due to:
> double rMax = 1.0;
>     while (resourceUsedWithWeightToResourceRatio(rMax, schedulables, type)
>         < totalResource) {
>       rMax *= 2.0;
>   }
> because resourceUsedWithWeightToResourceRatio will add minShare together.
> As i think is really should we bring in minShare when compute fairshare?
> My advice is we just consider weight is enough,and minshare's guarantee
> will get fulfill when assginContainer!
> Hope suggestion!



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message