hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ahmed Radwan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-140) Add capacity-scheduler-default.xml to provide a default set of configurations for the capacity scheduler.
Date Fri, 05 Oct 2012 19:56:03 GMT

    [ https://issues.apache.org/jira/browse/YARN-140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13470601#comment-13470601
] 

Ahmed Radwan commented on YARN-140:
-----------------------------------

Thanks Arun!

bq. It seems to me that the additional defaults are necessary only for unit tests - in which
case the correct fix is to make a copy of capacity-scheduler.xml into a test/resources folder
and get it on the classpath for our tests.

Yes, those additional ones are only for tests. I'll move the xml to a test/resources and have
it in the classpath for tests per you recommendation.

bq. Furthermore, please add a test-case to TestCapacitySchedulerConf to test for defaults.

The only remaining newly added default is the 100% capacity for root queue. I already modified
the TestCapapcityScheduler to verify that, and I'll check if any other tests are needed.

I'll upload an updated version of the patch.
                
> Add capacity-scheduler-default.xml to provide a default set of configurations for the
capacity scheduler.
> ---------------------------------------------------------------------------------------------------------
>
>                 Key: YARN-140
>                 URL: https://issues.apache.org/jira/browse/YARN-140
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: capacityscheduler
>            Reporter: Ahmed Radwan
>            Assignee: Ahmed Radwan
>         Attachments: YARN-140.patch, YARN-140_rev2.patch, YARN-140_rev3.patch, YARN-140_rev4.patch
>
>
> When setting up the capacity scheduler users are faced with problems like:
> {code}
> FATAL org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error starting ResourceManager
> java.lang.IllegalArgumentException: Illegal capacity of -1 for queue root
> {code}
> Which basically arises from missing basic configurations, which in many cases, there
is no need to explicitly provide, and a default configuration will be sufficient. For example,
to address the error above, the user need to add a capacity of 100 to the root queue.
> So, we need to add a capacity-scheduler-default.xml, this will be helpful to provide
the basic set of default configurations required to run the capacity scheduler. The user can
still override existing configurations or provide new ones in capacity-scheduler.xml. This
is similar to *-default.xml vs *-site.xml for yarn, core, mapred, hdfs, etc.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message