incubator-crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gabriel Reid <>
Subject Re: Moving PType and friends
Date Sun, 28 Oct 2012 20:12:37 GMT

On 28 Oct 2012, at 15:46, Matthias Friedrich <> wrote:

>>> Note: Java 8 will be a game changer; as far as I understand it, our
>>> abstract Fns are *not* going to work with Lambdas, we would need
>>> interfaces with only a single method. Does anyone think we have to
>>> address this before Java 8 is released?
>> I don't think we absolutely *need* to address this, but you can be sure that
>> I definitely do *want* to address it before the release of Java 8. Seeing the
>> way that pipelines can be implemented in Scrunch definitely makes me
>> jealous. It looks like we've got about 8 or 9 months to have this sorted out.
> The way I see it we have two options: a) change DoFn from an abstract
> class to a "functional interface" (that is, an interface that has
> exactly one abstract method), or b) provide some kind of mapper that
> takes a lambda expression and returns a DoFn.
> No matter which way we choose, we need to support both simple Fns that
> only implement process() (and are suitable for lambdas) and more
> complex Fns that are implemented as traditional Java classes. Complex
> Fns cannot be implemented as lambdas because they override
> initialize(), cleanup(), etc.
> When DoFn is a functional interface, we can work with Java EE style
> annotations to tag user-defined methods for cleanup/initialize and use
> dependency injection to pass in context.
> Big disclaimer: I've read a bit about lambdas but I haven't actually
> used them yet. I think we'll have to prototype extensively before
> making any decisions on this.

Yep, I definitely think that playing around with things and seeing what
works best is going to be the way to approach using lambdas. I've got
no experience with lambdas in Java either, but I'm definitely looking 
forward to seeing how we can fit them in Crunch.

- Gabriel

View raw message