hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sagar naik <sn...@attributor.com>
Subject When does Reduce job start
Date Tue, 04 Jan 2011 18:53:14 GMT
Hi All,

number  of map task: 1000s
number of reduce task:single digit

In such cases the reduce task wont  started even when few map task are
completed.
Example:
In my observation of a sample run of bin/hadoop jar
hadoop-*examples*.jar pi 10000 10, the reduce did not start untill 90%
of map task were complete.

The only reason, I can think of not starting  a reduce task is to
avoid the un-necessary transfer of map output data in case of
failures.


Is there a way to quickly start the reduce task in such case ?
Wht is the configuration param to change this behavior



Thanks,
Sagar

Mime
View raw message