hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: Setting number of mappers in Teragen
Date Wed, 26 Dec 2012 18:33:25 GMT
The MR1 teragen's mappers # depends on the total number of rows and
demanded # of maps.

How are you passing -Dmapred.map.tasks=20 (no spaces) exactly? All
generic options must go in before any other options do, so it should
appear right after the word "teragen" in your command.

On Wed, Dec 26, 2012 at 11:49 PM, anil gupta <anilgupta84@gmail.com> wrote:
> Hi All,
>
> I have 5 worker nodes and i have 4 map slots per node. So, i have 20 map
> slots in my cluster. But when, i start my Teragen job, it only spawns 2
> mappers for entire job. I have even tried using the option
> -Dmapred.map.tasks = 20 . Can anyone tell me how to force teragen to use 20
> mappers for generating the data? I am using cdh4.1.2 with Mapreducev1(Hadoop
> 0.20.2)
> --
> Thanks & Regards,
> Anil Gupta



-- 
Harsh J

Mime
View raw message