crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joseph Adler <joseph.ad...@gmail.com>
Subject Re: Focus of our next release?
Date Fri, 14 Sep 2012 18:30:18 GMT
It could be interesting to do the streaming on top of Apache Kafka because
both systems work well with Avro serialization.

On Fri, Sep 14, 2012 at 11:17 AM, Josh Wills <jwills@cloudera.com> wrote:

> I like the idea of having themes for releases. In my head, the theme of
> this release could be either
>
> a) Hacking the new MSCRPlanner code, esp. to add the ability to fuse
> different MSCR jobs into a single instance that it enables, or
> b) data access/integration points-- things like solr, hcatalog, hbase,
> cassandra, jdbc, etc. as input and output sources for Crunch pipelines, or
> c) API refactoring-- the crunch-api/crunch-impl/crunch-lib split, or
> d) working on a PStream API that would let people apply DoFns to streams
> and would build on top of things like WalMart's mupd8 or Storm or whatever.
>
> Of course, this is in addition to whatever fixes and new lib functions we
> want to add over time. I don't want anything heavyweight, but those are
> some of the larger-scale things that we'll need to tackle as a community,
> and I would think of completing each of those big things as corresponding
> to a release.
>
> Just my two cents.
>
> J
>
> On Fri, Sep 14, 2012 at 10:23 AM, Matthias Friedrich <matt@mafr.de> wrote:
>
> > Hi,
> >
> > should we discuss the focus of our next release? Maybe make a list
> > of things we want to achieve? Or would this be too much process?
> >
> > Regards,
> >   Matthias
> >
>
>
>
> --
> Director of Data Science
> Cloudera <http://www.cloudera.com>
> Twitter: @josh_wills <http://twitter.com/josh_wills>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message