hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karthik Kambatla (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4212) FairScheduler: Parent queues with 'Fair' policy should compute shares of all resources for its children during a recompute
Date Fri, 23 Oct 2015 15:23:27 GMT

    [ https://issues.apache.org/jira/browse/YARN-4212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14971152#comment-14971152
] 

Karthik Kambatla commented on YARN-4212:
----------------------------------------

Patch looks fairly straight-forward. Comments:
# Changes to FairScheduler.java seem spurious. Can we leave them out? 
# Test
## instead of editing a DRF-preemption test, can we add a new test - {{testAllocationWithMixedHierarchy}}

## Can we try multiple hierarchies outside of just root. May be, set root - fair, root.q1
- drf, root.q11 - fair, root.q2 - fair, root.q21 - drf, and verify jobs submitted to root.q11
and root.q21 get resources. Do we want to throw in FIFO as well somewhere in there? 

> FairScheduler: Parent queues with 'Fair' policy should compute shares of all resources
for its children during a recompute
> --------------------------------------------------------------------------------------------------------------------------
>
>                 Key: YARN-4212
>                 URL: https://issues.apache.org/jira/browse/YARN-4212
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Arun Suresh
>            Assignee: Arun Suresh
>              Labels: fairscheduler
>         Attachments: YARN-4212.1.patch
>
>
> The Fair Scheduler, while performing a {{recomputeShares()}} during an {{update()}} call,
uses the parent queues policy to distribute shares to its children.
> If the parent queues policy is 'fair', it only computes weight for memory and sets the
vcores fair share of its children to 0.
> Assuming a situation where we have 1 parent queue with policy 'fair' and multiple leaf
queues with policy 'drf', Any app submitted to the child queues with vcore requirement >
1 will always be above fairshare, since during the recomputeShare process, the child queues
were all assigned 0 for fairshare vcores.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message