hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Owen O'Malley" <omal...@apache.org>
Subject Re: Is continuous map reduce supported
Date Wed, 01 Sep 2010 16:56:34 GMT
On Wed, Sep 1, 2010 at 3:35 AM, Jeff Hammerbacher <hammer@cloudera.com> wrote:
>
> There have been several proposals for implementing such a feature.

Thanks for the great breakdown of the relevant work.

> As for actually being implemented in Hadoop MapReduce: the Apache project
> seems to have settled in to focus on stability rather than evolving new
> features.

That isn't true. We are actively adding new features. However, there
is certainly a focus on doing MapReduce well rather than trying to
implement all potential distributed computation paradigms. I suspect
that the right solution is doing two levels like Mesos:

http://www.eecs.berkeley.edu/Pubs/TechRpts/2010/EECS-2010-87.html

Where MapReduce is one framework running on the cluster. That way you
can keep the framework stable, support MapReduce well, and let other
users explore other idioms.

-- Owen

Mime
View raw message