hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yufei Gu (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (YARN-4212) FairScheduler: Parent queues is not allowed to be 'Fair' policy if its children have the "drf" policy
Date Thu, 19 Jan 2017 20:18:26 GMT

    [ https://issues.apache.org/jira/browse/YARN-4212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15830534#comment-15830534
] 

Yufei Gu edited comment on YARN-4212 at 1/19/17 8:17 PM:
---------------------------------------------------------

Thanks [~kasha] for the review. I've uploaded patch 007 for your comments.
# Remove both allowedParentPolicies and isParentPolicyAllowed, and add a non-static method
{{isChildPolicyAllowed}}, which is simpler.
# QueueManager
#- Rewrite the reinitSchedulingPolicies to recursively check all new policies first, then
set policies if there is no policy violation, otherwise don't set any polices for existing
queues. That is ALL or NONE. I also add a unit test for it.
#- Cannot put this logic into the recursive reinit in YARN-5831.



was (Author: yufeigu):
Thanks [~kasha] for the review. I've uploaded patch 007 for your comments.
# Remove both allowedParentPolicies and isParentPolicyAllowed, and add a non-static method
{{isChildPolicyAllowed}}, which is simpler.
# QueueManager
 - Rewrite the reinitSchedulingPolicies to recursively check all new policies first, then
set policies if there is no policy violation, otherwise don't set any polices for existing
queues. That is ALL or NONE. I also add a unit test for it.
 - Cannot put this logic into the recursive reinit in YARN-5831.


> FairScheduler: Parent queues is not allowed to be 'Fair' policy if its children have
the "drf" policy
> -----------------------------------------------------------------------------------------------------
>
>                 Key: YARN-4212
>                 URL: https://issues.apache.org/jira/browse/YARN-4212
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Arun Suresh
>            Assignee: Yufei Gu
>              Labels: fairscheduler
>         Attachments: YARN-4212.002.patch, YARN-4212.003.patch, YARN-4212.004.patch, YARN-4212.005.patch,
YARN-4212.006.patch, YARN-4212.007.patch, YARN-4212.1.patch
>
>
> The Fair Scheduler, while performing a {{recomputeShares()}} during an {{update()}} call,
uses the parent queues policy to distribute shares to its children.
> If the parent queues policy is 'fair', it only computes weight for memory and sets the
vcores fair share of its children to 0.
> Assuming a situation where we have 1 parent queue with policy 'fair' and multiple leaf
queues with policy 'drf', Any app submitted to the child queues with vcore requirement >
1 will always be above fairshare, since during the recomputeShare process, the child queues
were all assigned 0 for fairshare vcores.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message