hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun C Murthy (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-140) Add capacity-scheduler-default.xml to provide a default set of configurations for the capacity scheduler.
Date Fri, 05 Oct 2012 13:00:04 GMT

    [ https://issues.apache.org/jira/browse/YARN-140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13470281#comment-13470281
] 

Arun C Murthy commented on YARN-140:
------------------------------------

Ahmed, thanks, this is close - some feedback:

bq. To return the queue "root.default" as a child for root if no children are defined for
root.

I wouldn't do this. If someone has not defined root.child.queues it is a serious mis-configuration
- let's error out (as it happens today) and not try to 'guess'.

bq. To return a capacity of 100.0f for the queue "root.default" if it exists but no capacity
is specified for it. 

Again, for same reason as above, this represents a serious mis-configuration and I wouldn't
try to 'guess' capacities - error out is the right answer.

One more nit: Rather than delete/add capacity-scheduler.xml let's just do an 'svn mv' to preserve
history. 

Furthermore, please add a test-case to TestCapacitySchedulerConf to test for defaults.

----

It seems to me that the additional defaults are necessary only for unit tests - in which case
the correct fix is to make a copy of capacity-scheduler.xml into a test/resources folder and
get it on the classpath for our tests.


                
> Add capacity-scheduler-default.xml to provide a default set of configurations for the
capacity scheduler.
> ---------------------------------------------------------------------------------------------------------
>
>                 Key: YARN-140
>                 URL: https://issues.apache.org/jira/browse/YARN-140
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: capacityscheduler
>            Reporter: Ahmed Radwan
>            Assignee: Ahmed Radwan
>         Attachments: YARN-140.patch, YARN-140_rev2.patch, YARN-140_rev3.patch, YARN-140_rev4.patch
>
>
> When setting up the capacity scheduler users are faced with problems like:
> {code}
> FATAL org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error starting ResourceManager
> java.lang.IllegalArgumentException: Illegal capacity of -1 for queue root
> {code}
> Which basically arises from missing basic configurations, which in many cases, there
is no need to explicitly provide, and a default configuration will be sufficient. For example,
to address the error above, the user need to add a capacity of 100 to the root queue.
> So, we need to add a capacity-scheduler-default.xml, this will be helpful to provide
the basic set of default configurations required to run the capacity scheduler. The user can
still override existing configurations or provide new ones in capacity-scheduler.xml. This
is similar to *-default.xml vs *-site.xml for yarn, core, mapred, hdfs, etc.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message