hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bejoy KS" <bejoy.had...@gmail.com>
Subject Re: Set the number of maps
Date Thu, 01 Nov 2012 16:59:43 GMT
Hi Peter

'mapred.tasktracker.map.tasks.maximum' is not for setting an upper cap on the map tasks spawned
by a job. This property is used to set the map slots on each TaskTracker. It is TaskTracker
level property and cannot be overriden on a per job basis.

To control the number of map tasks for your job you need to increase the value for min and
max split sizes.

Regards
Bejoy KS

Sent from handheld, please excuse typos.

-----Original Message-----
From: Ted Dunning <tdunning@maprtech.com>
Date: Thu, 1 Nov 2012 09:50:10 
To: <user@hadoop.apache.org>
Reply-To: user@hadoop.apache.org
Subject: Re: Set the number of maps

Is the spelling of the option correct?

On Thu, Nov 1, 2012 at 6:43 AM, Cogan, Peter (Peter) <
Peter.Cogan@alcatel-lucent.com> wrote:

> Hi
>
> I understand that the maximum number of concurrent map tasks is set
> by mapred.tasktracker.map.tasks.maximum  - however I wish to run with a
> smaller number of maps (am testing disk IO). I thought that I could set
> that within the main program using
>
> conf.set("mapred.tasktracker.map.tasks.maximuma", "4");
>
>
> to run with 4 maps – but that seems to have no impact. I know I could just
> change the mapred-site.xml and restart map reduce but that's kind of a
> pain. Can it be set from within the code?
>
>
> Thanks
>
> Peter
>

Mime
View raw message