hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: Setting Number of Maps in 0.20.2
Date Tue, 16 Aug 2011 05:03:33 GMT

Setting map tasks is no longer allowed (as it doesn't make perfect
sense -- map tasks are determined based on your input files and the
file splits (hint: this is where you should be tweaking, if that's
your goal)). You can instead set a minimum split size to control your
map task parallelism: "mapred.min.split.size" as documented at

On Mon, Aug 15, 2011 at 2:20 AM, Jason Reed <jmreed3@asu.edu> wrote:
> I am trying to figure out how to set the number of maps to use in 0.20.2.
> If I was using JobConf in my program I could use:
> conf.setNumMapTasks(numMaps);
> However JobConf and the method are deprecated and when we started our
> project we structured everything to use Configuration and Job because of
> this. Is there a way to set the number of map tasks using Job and
> Configuration?
> Thank you for any help,
> Jason

Harsh J

View raw message