hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amir Youssefi" <am...@yahoo-inc.com>
Subject RE: Stop MR jobs after N records have been produced ?
Date Thu, 04 Sep 2008 23:18:25 GMT
Also see following jira for more discussions:

http://issues.apache.org/jira/browse/HADOOP-3973 

I will close this as new interface will address the issue.

- Amir 

-----Original Message-----
From: Owen O'Malley [mailto:owen.omalley@gmail.com] On Behalf Of Owen
O'Malley
Sent: Thursday, September 04, 2008 2:13 PM
To: core-user@hadoop.apache.org
Subject: Re: Stop MR jobs after N records have been produced ?


On Sep 4, 2008, at 12:48 PM, Tarandeep Singh wrote:

> Can I stop Map-Reduce jobs after mappers (or reducers) have produced N

> records ?

You could do this pretty easily by implementing a custom MapRunnable.  
There is no equivalent for reduces. The interface proposed in
HADOOP-1230 would support that kind of application. See:

http://svn.apache.org/repos/asf/hadoop/core/trunk/src/mapred/org/apache/
hadoop/mapreduce/

Look at the new Mapper and Reducer interfaces.

-- Owen

Mime
View raw message