Mailing-List: contact mapreduce-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: mapreduce-user@hadoop.apache.org
Received-SPF: neutral (nike.apache.org: local policy)
Message-Id: <E71B39A2-EC72-41D8-B068-74C20EDD3D69@yahoo-inc.com>
From: Arun C Murthy <acm@yahoo-inc.com>
To: mapreduce-user@hadoop.apache.org
In-Reply-To: <4CF2075D.7040601@gmail.com>
Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes
Content-Transfer-Encoding: 7bit
Subject: Re: delay the execution of reducers
Mime-Version: 1.0 (Apple Message framework v936)
Date: Sun, 28 Nov 2010 02:03:15 -0800
References: <4CF2075D.7040601@gmail.com>

Moving to mapreduce-user@, bcc common-user@. Please use project
specific lists.

mapreduce.reduce.slowstart.completed.maps is the right knob. Which  
version of hadoop are you running? If it isn't working, please open a  
jira. Thanks.

Arun

On Nov 27, 2010, at 11:40 PM, Da Zheng wrote:

> Hello,
>
> I found in Hadoop that reducers starts when a fraction of the number  
> of mappers
> is complete. However, in my case, I hope reducers to start only when  
> all mappers
> are complete. I searched for Hadoop configuration parameters, and  
> found
> mapred.reduce.slowstart.completed.maps, which seems to do what I  
> want. But no
> matter what value (0.99, 1.00, etc) I set to
> mapred.reduce.slowstart.completed.maps, reducers always start to  
> execute when
> about 10% of mappers are complete.
>
> Do I set the right parameter? Is there any other parameter I can use  
> for this
> purpose?
>
> Thanks,
> Da