hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Zhang <ezh...@yahoo-inc.com>
Subject speculative task execution and writing side-effect files
Date Wed, 23 Jan 2008 01:50:48 GMT
I tried to find more details on speculative task execution on hadoop 
site and mailing archive, but it didn't seem to get explained a lot.   
I'd appreciate if anybody can help me on following related questions:
1. In what situation would speculative task execution  kick in if it's 
enabled
2. how much performance gain we can generally expect from enabling of 
this feature. 
3. If I want to write out side-effect files named with unque names per 
task-attempt in the directory other than 
${mapred.output.dir}/_${taskid},  would framework discard files attemped 
by unsuccessful task attempts?
4. If I write files into subdirectories of 
${mapred.output.dir}/_${taskid} (e.g. 
${mapred.output.dir}/_${taskid}/${sub_dir}),  would framework take care 
of promoting ${sub_dir} to ${mapred.output.dir}?

Thanks a lot,

Eric

Mime
View raw message