hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karthik Kambatla (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4212) FairScheduler: Parent queues is not allowed to be 'Fair' policy if its children have the "drf" policy
Date Tue, 17 Jan 2017 18:14:26 GMT

    [ https://issues.apache.org/jira/browse/YARN-4212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15826533#comment-15826533

Karthik Kambatla commented on YARN-4212:

# SchedulingPolicy
## allowedParentPolicies should have a more specific return type, Set<Class<? extends
SchedulingPolicy>>? BTW, using Set<SchedulingPolicy> should not lead to too many
objects because the instances are cached in SchedulingPolicy. Alternatively, you could also
add a new field INSTANCE to each SchedulingPolicy. 
## Also, I wonder if isParentPolicyAllowed should be a static method. What do you think of
making it non-static, and getting rid of allowedParentPolicies altogether? 
# QueueManager
## In reinitSchedulingPolicies, shouldn't the parent-policy-check happen before setting the
queue's policy? 
## In another JIRA, you are making reinit recursive, should this be a part of that? 
## What if setting the queue policy at one level interferes with all child queues. For example,
assume "root.adhoc" is initially configured to be DRF. Subsequently, a number of queues have
been dynamically created all with DRF. Now, in a new configuration update, the admin sets
"root.adhoc" to FAIR. Should this configuration be loaded? And, any configuration should either
be loaded fully or not at all. May be, we need a test case for this as well.

> FairScheduler: Parent queues is not allowed to be 'Fair' policy if its children have
the "drf" policy
> -----------------------------------------------------------------------------------------------------
>                 Key: YARN-4212
>                 URL: https://issues.apache.org/jira/browse/YARN-4212
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Arun Suresh
>            Assignee: Yufei Gu
>              Labels: fairscheduler
>         Attachments: YARN-4212.002.patch, YARN-4212.003.patch, YARN-4212.004.patch, YARN-4212.005.patch,
YARN-4212.006.patch, YARN-4212.1.patch
> The Fair Scheduler, while performing a {{recomputeShares()}} during an {{update()}} call,
uses the parent queues policy to distribute shares to its children.
> If the parent queues policy is 'fair', it only computes weight for memory and sets the
vcores fair share of its children to 0.
> Assuming a situation where we have 1 parent queue with policy 'fair' and multiple leaf
queues with policy 'drf', Any app submitted to the child queues with vcore requirement >
1 will always be above fairshare, since during the recomputeShare process, the child queues
were all assigned 0 for fairshare vcores.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message