hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ravi Teja <ravit...@huawei.com>
Subject RE: Lost task tracker reschedules all tasktracker's successful map tasks
Date Tue, 27 Sep 2011 04:44:18 GMT
Hi Leonardo,

Yes, this is the default behavior. The Map tasks which are completed on the
Tasktracker are rescheduled , if its reducers are not completed. This is
because the Map output resides in the local machine, so if the Tasktracker
is down, the map output is also gone, which needs map task to be
re-executed. 
	If reducer finishes, even if TT is down, no rescheduling is done, as
final output is copied to the shared file system.	

Regards,
Ravi Teja

-----Original Message-----
From: Leonardo Gamas [mailto:leogamas@jusbrasil.com.br] 
Sent: Monday, September 26, 2011 11:11 PM
To: mapreduce-dev@hadoop.apache.org
Subject: Lost task tracker reschedules all tasktracker's successful map
tasks

Hi,

I have a very large MapReduce Job and sometimes a TaskTracker doesn't send a
heartbeat in the preconfigured amount of time, so it's considered dead. It's
ok, but all tasks already finished by this TaskTracker are lost too, or
better explained, are rescheduled and re-executed by another TaskTracker.

This is a default behavior or i'm experiencing some bug or miss
configuration?

My reguards,

Leonardo Gamas


Mime
View raw message