hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arun C Murthy <ar...@yahoo-inc.com>
Subject Re: Re-reduce, without re-map
Date Tue, 03 Apr 2007 10:06:30 GMT
Hi Mathijs,

Mathijs Homminga wrote:
> We have some troubles with the reduce phase of our job.
> Is it possible to re-execute the reduce tasks without the need to do all 
> map tasks again?

   That the MR-framework already does... you don't have to re-execute 
the maps for the *failed* reduces. Are you noticing something else?

   What are the 'troubles' you allude to? Also with once we get 
HADOOP-1127 in, you should try turing on 'speculative execution' - that 
helps when some tasks are very slow w.r.t other similar tasks.


> Thanks!
> Mathijs Homminga

View raw message