hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hemanth Yamijala (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-4178) The capacity scheduler's defaults for queues should be configurable.
Date Thu, 25 Sep 2008 13:25:45 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-4178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12634479#action_12634479
] 

Hemanth Yamijala commented on HADOOP-4178:
------------------------------------------

Comments:

capacity-scheduler.xml:
- Typo: "The default values would be applied to all the queues which don't" - should be "The
default values would be applied to all the queues which don't have"
- My understanding for default guaranteed capacity is that there will be *no* configuration
variable for the default. Instead, in code, we use -1 as the default value so that we know
when it is not defined. 

CapacitySchedulerConf:
- defaultGuaranteedCapacity is not needed.
- getGuaranteedCapacity should return -1 if the value is not configured. So, there is no need
to check if the value is not defined. The check for invalid values can ignore -1 and check
for other negatives and values > 100.
- The LOG variable is not used, and should be removed.
- Very minor nit: There are some extra lines after the variables for the defaults are declared.
- Should we check for sanity of other variables like reclaim time limit and minimum user limit
also ?

CapacityTaskScheduler:
- The computation of remaining capacities can be simpler, I think.
{code}
remainingCapacity = 100 - totalCapacity;
if (gcNotConfiguredQueues.size() > 0) {
  remainingCapacityPerQueue = Math.round(remainingCapacity / gcNotConfiguredQueues.size());
}
// distribute this for all queues in gcNotConfiguredQueues
{code}
Would this work ?
- I would recommend a variable name like queuesWithoutConfiguredGC instead of gcNotConfiguredQueues

Tests:
- testQueueWithUserDefinedDefaultProperties: The effective test would be to not define some
properties for queues, and verify that the overridden defaults are returned. This will be
similar to testQueueWithDefaultProperties.
- testGCAllocationToQueues:  one of the expected values is 18f, the others are 19f, shouldn't
they all be the same ?

> The capacity scheduler's defaults for queues should be configurable.
> --------------------------------------------------------------------
>
>                 Key: HADOOP-4178
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4178
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: contrib/capacity-sched
>    Affects Versions: 0.19.0
>            Reporter: Owen O'Malley
>            Assignee: Sreekanth Ramakrishnan
>            Priority: Blocker
>             Fix For: 0.19.0
>
>         Attachments: HADOOP-4178-1.patch, HADOOP-4178-2.patch, HADOOP-4178-3.patch, HADOOP-4178-4.patch
>
>
> The default values for the queue attributes should be configurable rather than hard coded.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message