hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: Setting Number of Maps in 0.20.2
Date Tue, 16 Aug 2011 05:03:33 GMT
Jason,

Setting map tasks is no longer allowed (as it doesn't make perfect
sense -- map tasks are determined based on your input files and the
file splits (hint: this is where you should be tweaking, if that's
your goal)). You can instead set a minimum split size to control your
map task parallelism: "mapred.min.split.size" as documented at
http://hadoop.apache.org/common/docs/r0.20.2/api/org/apache/hadoop/mapreduce/InputFormat.html

On Mon, Aug 15, 2011 at 2:20 AM, Jason Reed <jmreed3@asu.edu> wrote:
> I am trying to figure out how to set the number of maps to use in 0.20.2.
>
> If I was using JobConf in my program I could use:
>
> conf.setNumMapTasks(numMaps);
>
> However JobConf and the method are deprecated and when we started our
> project we structured everything to use Configuration and Job because of
> this. Is there a way to set the number of map tasks using Job and
> Configuration?
>
> Thank you for any help,
> Jason
>



-- 
Harsh J

Mime
View raw message