hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-4789) Change fair scheduler to share between pools by default, not between invidual jobs
Date Sun, 14 Dec 2008 11:08:46 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-4789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12656378#action_12656378

Hadoop QA commented on HADOOP-4789:

-1 overall.  Here are the results of testing the latest attachment 
  against trunk revision 726129.

    +1 @author.  The patch does not contain any @author tags.

    +1 tests included.  The patch appears to include 3 new or modified tests.

    +1 javadoc.  The javadoc tool did not generate any warning messages.

    +1 javac.  The applied patch does not increase the total number of javac compiler warnings.

    +1 findbugs.  The patch does not introduce any new Findbugs warnings.

    +1 Eclipse classpath. The patch retains Eclipse classpath integrity.

    -1 core tests.  The patch failed core unit tests.

    +1 contrib tests.  The patch passed contrib unit tests.

Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3742/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3742/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3742/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3742/console

This message is automatically generated.

> Change fair scheduler to share between pools by default, not between invidual jobs
> ----------------------------------------------------------------------------------
>                 Key: HADOOP-4789
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4789
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: contrib/fair-share
>            Reporter: Matei Zaharia
>            Assignee: Matei Zaharia
>            Priority: Minor
>         Attachments: hadoop-4789-v1.patch
> The fair scheduler currently treats jobs as equal entities in sharing by default, so
that a user who submits 2 jobs gets 2x the share of a user who submits only 1 job. We found
that it makes more sense to support equal shares between individual pools instead, and have
one pool per user, because users can otherwise game the system by submitting multiple small
jobs. This patch will set the scheduler to share between pools by default and set the default
pool assignment process to one pool per user. it will also be possible to give weights to
pools so that some users/groups/applications get a larger share of the cluster if they really
do need to run more jobs.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message