hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: How to set the max mappers per node on a per-job basis?
Date Sat, 31 May 2014 03:49:50 GMT
This has been discussed in past. There is no current dynamic way to
control the parallel execution on a per-node basis.

Scheduler configurations will let you control overall parallelism (#
of simultaneous tasks) of specific jobs on a cluster-level basis, but
not on a per-node level.

On Sat, May 31, 2014 at 4:08 AM, jeremy p
<athomewithagroovebox@gmail.com> wrote:
> Hello all,
>
> I have two jobs, Job A and Job B.  Job A is not very CPU-intensive, and so
> we would like to run it with 50 mappers per node.  Job B is very
> CPU-intensive, and so we would like to run it with 25 mappers per node.  How
> can we request a different number of mappers per node for each job?  From
> what I've read, mapred.tasktracker.map.tasks.maximum and
> mapred.tasktracker.reduce.tasks.maximum cannot be overridden from the
> client.
>
> --Jeremy



-- 
Harsh J

Mime
View raw message