hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ravi Gummadi <gr...@yahoo-inc.com>
Subject Re: Custom reduce progress reporting
Date Thu, 21 Oct 2010 05:50:26 GMT
Currently, they are hardcoded in sourcecode. Please see 
Progress.addPhase() and Progress.addPhase(String status, float 
weightage) and their calls in MapTask.run() and ReduceTask.run() for 
changing the weightages of phases. In ReduceTask.run(), change the calls 
from addPhase() to addPhase(status, weightage) for all 3 phases with 
different weightages(like 0.4, 0.25, 0.35 for the 3 phases).

-Ravi

Ed Mazur wrote:
> Is there a configuration property or some other mechanism for changing
> how reduce tasks report progress? E.g., instead of the shuffle, merge,
> and reduce each counting for 33%, can this be changed?
>
> The Hadoop Online Prototype did this, but I can't figure out how.
>
> Ed
>   


Mime
View raw message