hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sundeep Kambhampati <kambh...@cse.ohio-state.edu>
Subject Fault tolerance and Speculative Execution
Date Thu, 18 Jul 2013 17:02:50 GMT
Hi all,
Is it true that Hadoop 'always' starts same map tasks multiple times in 
order to be fault tolerant. i.e. same task is launched on several 
machines so that even if a node fails then same task would be available 
on other node. And in case no node fails redundant task that finishes 
late is killed. If it is true how can I change that configuration for 
Hadoop to do it or not do it.

Speculative execution on the other hand does what I explained above 
(redundant map tasks) but only after all the map tasks are scheduled and 
if some nodes are free it starts redundant map tasks for those which are 
running slow. Is it always true? How do change this configuration 
enable/disable.

I am using Hadoop-1.1.2 incase version matters.

I really appreciate if someone could help me with this. Thank you.

Regards
Sundeep



Mime
View raw message