hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karthik Kambatla (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (YARN-4702) FairScheduler: Allow setting maxResources for ad hoc queues
Date Wed, 17 Aug 2016 21:30:21 GMT

    [ https://issues.apache.org/jira/browse/YARN-4702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15425413#comment-15425413
] 

Karthik Kambatla edited comment on YARN-4702 at 8/17/16 9:30 PM:
-----------------------------------------------------------------

Thanks for the prompt updates, Daniel. The latest patch looks pretty good. Some minor comments:
# FairScheduer
## Sprious import
## applyChildDefaults should be called before maxRunningEnforcer.updateRunnabilityOnReload().
May be we should a test case for checking runnability of child queues in a follow-up JIRA?

## applyChildDefaults: For the temporary sets of leaf and parent queues, can we include "configured"
in the variable names? 
## applyChildDefaults: If a queue is ad hoc, shouldn't we be checking the maxChildResources
of the parent queue?
# There appears to be some duplication between {{FairScheduler#applyChildDefaults}} and {{QueueManager#setChildResourceLimits}}.
Is the latter necessary? In any case, at the very least, we should have a helper method in
AllocationConfiguration called {{isConfiguredQueue(String queueName)}} so both methods could
use it. 
# TestFairScheduler
## testFairShareWithLowMaxResources: Setting the weight to .5 for both queues is as good as
setting it to 1, which is as good as not setting it. Can we leave out the weight setting altogether?





was (Author: kasha):
Thanks for the prompt updates, Daniel. The latest patch looks pretty good. Some minor comments:
# FairScheduer
## Sprious import
## applyChildDefaults should be called before maxRunningEnforcer.updateRunnabilityOnReload().
May be we should a test case for checking runnability of child queues in a follow-up JIRA?

## applyChildDefaults: For the temporary sets of leaf and parent queues, can we include "configured"
in the variable names? 
## applyChildDefaults: If a queue is ad hoc, shouldn't we be checking the maxChildResources
of the parent queue?
# There appears to be some duplication between FairScheduler#applyChildDefaults and QueueManager#setChildResourceLimits.
Is the latter necessary? In any case, at the very least, we should have a helper method in
{{AllocationConfiguration called isConfiguredQueue(String queueName)}} so both methods could
use it. 
# TestFairScheduler
## testFairShareWithLowMaxResources: Setting the weight to .5 for both queues is as good as
setting it to 1, which is as good as not setting it. Can we leave out the weight setting altogether?




> FairScheduler: Allow setting maxResources for ad hoc queues
> -----------------------------------------------------------
>
>                 Key: YARN-4702
>                 URL: https://issues.apache.org/jira/browse/YARN-4702
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: fairscheduler
>    Affects Versions: 2.8.0
>            Reporter: Karthik Kambatla
>            Assignee: Daniel Templeton
>         Attachments: YARN-4702.001.patch, YARN-4702.002.patch, YARN-4702.003.patch, YARN-4702.004.patch,
YARN-4702.005.patch, YARN-4702.006.patch
>
>
> FairScheduler allows creating queues on the fly. Unlike other queues, one cannot set
maxResources for these queues since they don't show up in the xml config. Adding a default
maxResources for ad hoc pools would help. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message