hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karthik Kambatla (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4212) FairScheduler: Parent queues is not allowed to be 'Fair' policy if its children have the "drf" policy
Date Tue, 05 Jul 2016 23:55:11 GMT

    [ https://issues.apache.org/jira/browse/YARN-4212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15363498#comment-15363498
] 

Karthik Kambatla commented on YARN-4212:
----------------------------------------

Took a quick look at the patch. 

One high-level question: instead of checking for parents in AllocationFileLoaderService, would
it make sense to check for the parent compatibility while creating the queue? That way, we
would be able to handle dynamically created queues in addition to those created using the
allocation file. Also, we will not have to include a separate map to keep all the parents.


In general, I am not a fan of the current approach of having all those maps with information
about queues outside of FSQueue hierarchy. Having all the information regarding a queue in
one place seems the right way to go. I know [~rchiang] was interested in improving this. 

> FairScheduler: Parent queues is not allowed to be 'Fair' policy if its children have
the "drf" policy
> -----------------------------------------------------------------------------------------------------
>
>                 Key: YARN-4212
>                 URL: https://issues.apache.org/jira/browse/YARN-4212
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Arun Suresh
>            Assignee: Yufei Gu
>              Labels: fairscheduler
>         Attachments: YARN-4212.002.patch, YARN-4212.003.patch, YARN-4212.004.patch, YARN-4212.1.patch
>
>
> The Fair Scheduler, while performing a {{recomputeShares()}} during an {{update()}} call,
uses the parent queues policy to distribute shares to its children.
> If the parent queues policy is 'fair', it only computes weight for memory and sets the
vcores fair share of its children to 0.
> Assuming a situation where we have 1 parent queue with policy 'fair' and multiple leaf
queues with policy 'drf', Any app submitted to the child queues with vcore requirement >
1 will always be above fairshare, since during the recomputeShare process, the child queues
were all assigned 0 for fairshare vcores.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message