incubator-crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Wills <>
Subject Re: Focus of our next release?
Date Fri, 14 Sep 2012 18:17:30 GMT
I like the idea of having themes for releases. In my head, the theme of
this release could be either

a) Hacking the new MSCRPlanner code, esp. to add the ability to fuse
different MSCR jobs into a single instance that it enables, or
b) data access/integration points-- things like solr, hcatalog, hbase,
cassandra, jdbc, etc. as input and output sources for Crunch pipelines, or
c) API refactoring-- the crunch-api/crunch-impl/crunch-lib split, or
d) working on a PStream API that would let people apply DoFns to streams
and would build on top of things like WalMart's mupd8 or Storm or whatever.

Of course, this is in addition to whatever fixes and new lib functions we
want to add over time. I don't want anything heavyweight, but those are
some of the larger-scale things that we'll need to tackle as a community,
and I would think of completing each of those big things as corresponding
to a release.

Just my two cents.


On Fri, Sep 14, 2012 at 10:23 AM, Matthias Friedrich <> wrote:

> Hi,
> should we discuss the focus of our next release? Maybe make a list
> of things we want to achieve? Or would this be too much process?
> Regards,
>   Matthias

Director of Data Science
Cloudera <>
Twitter: @josh_wills <>

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message