hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Amar Kamat <ama...@yahoo-inc.com>
Subject Re: Reduce Hangs
Date Thu, 27 Mar 2008 14:50:44 GMT
On Thu, 27 Mar 2008, Natarajan, Senthil wrote:

> Hi,
> I have small Hadoop cluster, one master and three slaves.
> When I try the example wordcount on one of our log file (size ~350 MB)
> Map runs fine but reduce always hangs (sometime around 19%,60% ...) after very long time
it finishes.
> I am seeing this error
> Shuffle Error: Exceeded MAX_FAILED_UNIQUE_FETCHES; bailing-out
This error occurs when the reducer fails to fetch map-task-output from 5
unique map tasks. Before considering an attempt as failed the reducer
tries to fetch the map output for 7 times in 5 mins (default config).
In case of the job failure check the following
1. Is this problem common in all the reducers?
2. Are the map tasks same across all the reducers for which the failure is
3. Is there atleast one map task whose output is successfully fetched?
If the job becomes successful then there might be some problem with the
> In the log I am seeing this
> INFO org.apache.hadoop.mapred.TaskTracker: task_200803261535_0001_r_000000_0 0.18333334%
reduce > copy (11 of 20 at 0.02 MB/s) >
> Do you know what might be the problem.
> Thanks,
> Senthil

View raw message