hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Goel, Ankur" <ankur.g...@corp.aol.com>
Subject RE: How can I control Number of Mappers of a job?
Date Thu, 31 Jul 2008 05:17:53 GMT
How big is your cluster? Assuming you are running a single node cluster,

Hadoop-default.xml has a parameter 'mapred.map.tasks' that is set to 2.
So
By default, no matter how many map tasks are calculated by framework,
only  2 map task will execute on a single node cluster.

-----Original Message-----
From: Gopal Gandhi [mailto:gopal.gandhi2008@yahoo.com] 
Sent: Thursday, July 31, 2008 4:38 AM
To: core-user@hadoop.apache.org
Cc: core-dev@hadoop.apache.org
Subject: How can I control Number of Mappers of a job?

The motivation is to control the max # of mappers of a job. For example,
the input data is 246MB, divided by 64M is 4. If by default there will
be 4 mappers launched on the 4 blocks. 
What I want is to set its max # of mappers as 2, so that 2 mappers are
launched first and when they completes on the first 2 blocks, another 2
mappers start on the rest 2 blocks. Does Hadoop provide a way?



      

Mime
View raw message