hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <qwertyman...@gmail.com>
Subject Re: Is continuous map reduce supported
Date Tue, 24 Aug 2010 18:05:24 GMT
There's Chain-Mapping and Chain-Reducing available. With good docs:
http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/mapred/lib/ChainReducer.html

However, something as simple as Twister (which has iterative
mapreduces based on a while-like condition loop) isn't directly
available. One sometimes needs to chain jobs together to achieve this
in pure-Hadoop.

Projects like Hive, Pig, and Cascading help with this a bit (plan
building, optimization of plan, execution, etc.).

On Tue, Aug 24, 2010 at 10:25 PM, Stephen Mullins <smullins7@gmail.com> wrote:
> Hello,
>
> I have not used Hadoop but am researching it for an analytics project. I
> would like to know if Hadoop supports continuous or incremental map reduce
> functionality. If not, are there any plans to add it?
>
> Thanks,
> Stephen
>



-- 
Harsh J
www.harshj.com

Mime
View raw message