hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matei Zaharia (JIRA)" <j...@apache.org>
Subject [jira] Updated: (MAPREDUCE-546) Provide sample fair scheduler config file in conf/ and set config file property to point to this by default
Date Mon, 06 Jul 2009 22:47:14 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Matei Zaharia updated MAPREDUCE-546:

    Attachment: mapreduce-546.patch

Here's a patch for this issue that makes the scheduler use fair-scheduler.xml off the classpath
if no other allocation file is specified through mapred.fairscheduler.allocation.file. (We
keep this parameter for backwards compatibility).

The only tricky part was using the URL from the ClassLoader's getResource method instead of
a String for the path to the file. I followed the example set in Configuration of having an
Object as the allocation file name that may be either an URL or a String, because I didn't
want to force users of the existing jobconf parameter to supply a file:// URL, and I also
didn't want to append file:// in front (this doesn't work with relative paths, and although
the docs have said to use an absolute path since the scheduler was released, it could be confusing).

I haven't included a unit test because the code changes are minor and I can't think of an
easy way to unit test this. However, I did manually test that configurations are found whether
the default config file fair-scheduler.xml is used or another file is specified through the
config parameter, and also that the allocation file is reloaded at runtime in both situations.

I've also updated the docs to reflect the new functionality and emptied-out the fair-scheduler.xml
template so that it creates no pools by default. The docs cover all the features that were
described in the old fair-scheduler.xml.template.

> Provide sample fair scheduler config file in conf/ and set config file property to point
to this by default
> -----------------------------------------------------------------------------------------------------------
>                 Key: MAPREDUCE-546
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-546
>             Project: Hadoop Map/Reduce
>          Issue Type: New Feature
>            Reporter: Matei Zaharia
>            Priority: Minor
>             Fix For: 0.21.0
>         Attachments: mapreduce-546.patch
> The capacity scheduler includes a config file template in hadoop/conf, so it would make
sense to create a similar one for the fair scheduler and mention it in the README.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message