hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ahmed Radwan (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-140) Add capacity-scheduler-default.xml to provide a default set of configurations for the capacity scheduler.
Date Fri, 05 Oct 2012 08:40:49 GMT

     [ https://issues.apache.org/jira/browse/YARN-140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Ahmed Radwan updated YARN-140:

    Attachment: YARN-140_rev4.patch

The cause of these failures seems to be a couple of missing default configurations that were
previously loaded from capacity-scheduler.xml. I have added such defaults to CapacitySchedulerConfiguration.java.
I have ran all these failing tests locally, and all of them succeed except two  (TestApplicationCleanup
& TestGetGroups) which fail with and without the patch, so they seem to be unrelated to
this patch.

The two additional defaults that I needed to add to fix these tests are:

1- To return the queue "root.default" as a child for root if no children are defined for root.
2- To return a capacity of 100.0f for the queue "root.default" if it exists but no capacity
is specified for it. (I was unsure about this default considering the case when root has other
children specified, but I think it should be fine since both undefined capacity (i.e., -1)
or maximum capacity (i.e., 100) will represent an invalid configuration.

I am attaching the updated patch.
> Add capacity-scheduler-default.xml to provide a default set of configurations for the
capacity scheduler.
> ---------------------------------------------------------------------------------------------------------
>                 Key: YARN-140
>                 URL: https://issues.apache.org/jira/browse/YARN-140
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: capacityscheduler
>            Reporter: Ahmed Radwan
>            Assignee: Ahmed Radwan
>         Attachments: YARN-140.patch, YARN-140_rev2.patch, YARN-140_rev3.patch, YARN-140_rev4.patch
> When setting up the capacity scheduler users are faced with problems like:
> {code}
> FATAL org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error starting ResourceManager
> java.lang.IllegalArgumentException: Illegal capacity of -1 for queue root
> {code}
> Which basically arises from missing basic configurations, which in many cases, there
is no need to explicitly provide, and a default configuration will be sufficient. For example,
to address the error above, the user need to add a capacity of 100 to the root queue.
> So, we need to add a capacity-scheduler-default.xml, this will be helpful to provide
the basic set of default configurations required to run the capacity scheduler. The user can
still override existing configurations or provide new ones in capacity-scheduler.xml. This
is similar to *-default.xml vs *-site.xml for yarn, core, mapred, hdfs, etc.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message