hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alex Munteanu <a...@geraeteturnen.com>
Subject Fwd: how to set max map tasks individually for each job?
Date Thu, 03 Jun 2010 08:45:26 GMT

I am running several different mapreduce jobs. For some of them it is
better to have a rather high number of running map tasks per node,
whereas others do very intensive read operations on our database
resulting in read timeouts. So for these jobs I'd like to set a much
smaller limit of concurrently running map tasks.

I tried to overwrite the "mapred.tasktracker.map.tasks.maximum" value in
our job setup but it seems to be a global setting since it affects the
tasktrackers, not the scheduling component.
Also i've found https://issues.apache.org/jira/browse/HADOOP-5170 on the
web. It seems to be exactly what I need but the changes seem not to be
in the current 0.20.2 release which I am using and they also seem to
involve the JobConf class which for now is marked deprecated.

So I have no idea how to do this without changing the global tasktracker map task maximum
value and
restarting the system.


View raw message