hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arun C Murthy <...@yahoo-inc.com>
Subject Re: delay the execution of reducers
Date Sun, 28 Nov 2010 10:03:15 GMT
Moving to mapreduce-user@, bcc common-user@. Please use project
specific lists.

mapreduce.reduce.slowstart.completed.maps is the right knob. Which  
version of hadoop are you running? If it isn't working, please open a  
jira. Thanks.

Arun

On Nov 27, 2010, at 11:40 PM, Da Zheng wrote:

> Hello,
>
> I found in Hadoop that reducers starts when a fraction of the number  
> of mappers
> is complete. However, in my case, I hope reducers to start only when  
> all mappers
> are complete. I searched for Hadoop configuration parameters, and  
> found
> mapred.reduce.slowstart.completed.maps, which seems to do what I  
> want. But no
> matter what value (0.99, 1.00, etc) I set to
> mapred.reduce.slowstart.completed.maps, reducers always start to  
> execute when
> about 10% of mappers are complete.
>
> Do I set the right parameter? Is there any other parameter I can use  
> for this
> purpose?
>
> Thanks,
> Da


Mime
View raw message