hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Bieniosek <mich...@powerset.com>
Subject Re: Question on running simultaneous jobs
Date Wed, 09 Jan 2008 20:50:48 GMT
Hadoop-0.14 introduced job priorities (https://issues.apache.org/jira/ 
browse/HADOOP-1433); you might be able to get somewhere with this.

Another possibility is to create two mapreduce clusters on top of the  
same dfs cluster.

The mapred.tasktracker.tasks.maximum doesn't do what you think -- it  
actually controls the numbers of tasks that run simultaneously on a  
tasktracker machine.

-Michael

On Jan 9, 2008, at 11:25 AM, Xavier Stevens wrote:

> Does Hadoop support running simultaneous jobs?  If so, what parameters
> do I need to set in my job configuration?  We basically want to give a
> job that takes a really long time, half of the total resources of the
> cluster so other jobs don't queue up behind it.
>
> I am using Hadoop 0.14.2 currently.  I tried setting
> mapred.tasktracker.tasks.maximum to be half of the maximum  
> specified in
> mapred-default.xml.  This shows the change in the web administration
> page for the job, but it has no effect on the actual numbers of tasks
> running.
>
> Thanks,
>
> Xavier
>


Mime
View raw message