hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Leonardo Gamas <leoga...@jusbrasil.com.br>
Subject Re: Lost task tracker reschedules all tasktracker's successful map tasks
Date Thu, 29 Sep 2011 16:59:57 GMT
No, the reducers are fine, or at least i didn't observe any problem.

The question is: the intermediary (before any reducer) results of completed
individual tasks are recorded in the HDFS, right? So why are these results
discarded, since the lost of the tasktracker is not the lost of already
processed data?

--Leonardo Gamas

2011/9/29 Robert Evans <evans@yahoo-inc.com>

>  If a TaskTracker is lost then it cannot serve up any Map results to
> Reducers that will need them so the Map tasks have to be rerun.  I am not
> sure if this is the behavior you are seeing or not.  Are completed Reducers
> being rerun as well?
>
> --Bobby Evans
>
>
> On 9/29/11 11:15 AM, "Leonardo Gamas" <leogamas@jusbrasil.com.br> wrote:
>
> Hi,
>
> I have a very large MapReduce Job and sometimes a TaskTracker doesn't send
> a heartbeat in the preconfigured amount of time, so it's considered dead.
> It's ok, but all tasks already finished by this TaskTracker are lost too, or
> better explained, are rescheduled and re-executed by another TaskTracker.
>
> This is a default behavior or i'm experiencing some bug or miss
> configuration?
>
> My reguards,
>
> Leonardo Gamas
>
>
>

Mime
View raw message