hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alan Heirich (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-707) Provide a jobconf property for explicitly assigning a job to a pool
Date Thu, 05 Nov 2009 17:01:33 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12773973#action_12773973
] 

Alan Heirich commented on MAPREDUCE-707:
----------------------------------------

This Hudson result is frustrating.  TestGridmixSubmission passes when I run it in my workspace
on Mac OS X.  I cannot reproduce the failure.  But Hudson compains that testSubmit fails!
 The diagnostic is:
Error Message

Mismatched output bytes 1737801/1764471
Stacktrace

junit.framework.AssertionFailedError: Mismatched output bytes 1737801/1764471
	at org.apache.hadoop.mapred.gridmix.TestGridmixSubmission$TestMonitor.check(TestGridmixSubmission.java:231)
	at org.apache.hadoop.mapred.gridmix.TestGridmixSubmission$TestMonitor.verify(TestGridmixSubmission.java:140)
	at org.apache.hadoop.mapred.gridmix.TestGridmixSubmission$DebugGridmix.checkMonitor(TestGridmixSubmission.java:263)
	at org.apache.hadoop.mapred.gridmix.TestGridmixSubmission.testSubmit(TestGridmixSubmission.java:297

> Provide a jobconf property for explicitly assigning a job to a pool
> -------------------------------------------------------------------
>
>                 Key: MAPREDUCE-707
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-707
>             Project: Hadoop Map/Reduce
>          Issue Type: New Feature
>          Components: contrib/fair-share
>            Reporter: Matei Zaharia
>            Priority: Trivial
>         Attachments: MAPREDUCE-707-1-apache.patch, MAPREDUCE-707-2-apache.patch, MAPREDUCE-707-apache.patch
>
>
> A common use case of the fair scheduler is to have one pool per user, but then to define
some special pools for various production jobs, import jobs, etc. Therefore, it would be nice
if jobs went by default to the pool of the user who submitted them, but there was a setting
to explicitly place a job in another pool. Today, this can be achieved through a sort of trick
in the JobConf:
> {code}
> <property>
>   <name>mapred.fairscheduler.poolnameproperty</name>
>   <value>pool.name</value>
> </property>
> <property>
>   <name>pool.name</name>
>   <value>${user.name}</value>
> </property>
> {code}
> This JIRA proposes to add a property called mapred.fairscheduler.pool that allows a job
to be placed directly into a pool, avoiding the need for this trick.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message