hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yufei Gu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4212) FairScheduler: Parent queues with 'Fair' policy should compute shares of all resources for its children during a recompute
Date Mon, 23 May 2016 21:16:12 GMT

    [ https://issues.apache.org/jira/browse/YARN-4212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15297096#comment-15297096
] 

Yufei Gu commented on YARN-4212:
--------------------------------

The way to calculate the vcore in fair policy might not be a good idea. After discussion with
[~kasha], we think the best solution is to force the policy of parents to be drf when policy
of children are drf. So we decide to do one more sanity check while loading the allocation
configure file. [~kasha] suggested a more systematic sanity check by having a function that
returns allowed parent policies based on the input policy. For example, if one queue is fair,
its parent queue policy should be fair or drf. The complete relationships will be the following:
- drf: drf
- fair: fair, drf
- fifo: fair, drf


> FairScheduler: Parent queues with 'Fair' policy should compute shares of all resources
for its children during a recompute
> --------------------------------------------------------------------------------------------------------------------------
>
>                 Key: YARN-4212
>                 URL: https://issues.apache.org/jira/browse/YARN-4212
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Arun Suresh
>            Assignee: Yufei Gu
>              Labels: fairscheduler
>         Attachments: YARN-4212.1.patch
>
>
> The Fair Scheduler, while performing a {{recomputeShares()}} during an {{update()}} call,
uses the parent queues policy to distribute shares to its children.
> If the parent queues policy is 'fair', it only computes weight for memory and sets the
vcores fair share of its children to 0.
> Assuming a situation where we have 1 parent queue with policy 'fair' and multiple leaf
queues with policy 'drf', Any app submitted to the child queues with vcore requirement >
1 will always be above fairshare, since during the recomputeShare process, the child queues
were all assigned 0 for fairshare vcores.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message