hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: Multiple reduce task retries running at same time
Date Tue, 29 Jan 2013 16:48:14 GMT
Hi Ben,

Take a look at the Speculative Execution feature of MR which should
answer your question. See its section under 'Fault Tolerance' here:
http://developer.yahoo.com/hadoop/tutorial/module4.html#tolerence.

On Tue, Jan 29, 2013 at 1:08 PM, Ben Kim <benkimkimben@gmail.com> wrote:
> Attached a screenshot showing the retries
>
>
> On Tue, Jan 29, 2013 at 4:35 PM, Ben Kim <benkimkimben@gmail.com> wrote:
>>
>> Hi!
>>
>> I have come across the situation where i found a single reducer task
>> executing with multiple retries simultaneously.
>> Which is potent for slowing down the whole reduce process for large data
>> sets.
>>
>> Is this pretty normal to yall for hadoop 1.0.3?
>>
>> --
>>
>> Benjamin Kim
>> benkimkimben at gmail
>
>
>
>
> --
>
> Benjamin Kim
> benkimkimben at gmail



-- 
Harsh J

Mime
View raw message