flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Till Rohrmann <till.rohrm...@gmail.com>
Subject Re: Adding a serial collection-based execution mode
Date Tue, 23 Sep 2014 08:54:21 GMT
Great idea. +1
On Sep 23, 2014 9:28 AM, "Fabian Hueske" <fhueske@apache.org> wrote:

> +1
>
> 2014-09-23 7:03 GMT+02:00 Aljoscha Krettek <aljoscha@apache.org>:
>
> > Sounds like a good Idea. But why only "new test programs"? The
> > existing ones would also run on Collections, right?
> >
> > On Tue, Sep 23, 2014 at 1:28 AM, Stephan Ewen <sewen@apache.org> wrote:
> > > What do you think of changing the TestBase such that all new test
> > programs
> > > are not only executed with the LocalExecutor, but also with the
> > Collection
> > > execution environment? Would give us massive coverage for the
> collection
> > > execution for free...
> > >
> > > On Mon, Sep 22, 2014 at 11:09 PM, Stephan Ewen <sewen@apache.org>
> wrote:
> > >
> > >> Will do mapPartition...
> > >>
> > >> On Mon, Sep 22, 2014 at 7:34 PM, Ufuk Celebi <uce@apache.org> wrote:
> > >>
> > >>>
> > >>> On 22 Sep 2014, at 19:08, Stephan Ewen <sewen@apache.org> wrote:
> > >>>
> > >>> > I'll take "cross". Need it for tests...
> > >>> >
> > >>> > On Mon, Sep 22, 2014 at 5:48 PM, Stephan Ewen <sewen@apache.org>
> > wrote:
> > >>> >
> > >>> >> Hi!
> > >>> >>
> > >>> >> I added a new branch (collections2) that includes the commit
> moving
> > the
> > >>> >> typeutils to "flink-core" and including the code for the runtime
> > >>> context.
> > >>> >>
> > >>> >> Those who need any of that, please rebase your branch...
> > >>> >>
> > >>> >> Stephan
> > >>> >>
> > >>> >>
> > >>> >> On Mon, Sep 22, 2014 at 4:33 PM, Aljoscha Krettek <
> > aljoscha@apache.org
> > >>> >
> > >>> >> wrote:
> > >>> >>
> > >>> >>> I'll do groupReduce
> > >>> >>>
> > >>> >>> On Mon, Sep 22, 2014 at 4:13 PM, Till Rohrmann <
> > >>> till.rohrmann@gmail.com>
> > >>> >>> wrote:
> > >>> >>>> Then I'll take the join operation.
> > >>> >>>>
> > >>> >>>> On Mon, Sep 22, 2014 at 3:44 PM, Ufuk Celebi <uce@apache.org>
> > wrote:
> > >>> >>>>
> > >>> >>>>>
> > >>> >>>>> On 22 Sep 2014, at 15:37, Stephan Ewen <sewen@apache.org>
> wrote:
> > >>> >>>>>
> > >>> >>>>>> Hi everyone!
> > >>> >>>>>>
> > >>> >>>>>> I have started with a patch that introduces
*"local
> > >>> collection-based
> > >>> >>>>>> execution"* of Flink programs.
> > >>> >>>>>
> > >>> >>>>> Very nice!
> > >>> >>>>>
> > >>> >>>>> To have a list of missing operators:
> > >>> >>>>>
> > >>> >>>>> - flatMap <= I'll take this
> > >>> >>>>> - filter <= I'll take this
> > >>> >>>>> - reduce <= I'll take this
> > >>> >>>>> - groupreduce
> > >>> >>>>> - cogroup
> > >>> >>>>> - join
> > >>> >>>>>
> > >>>
> > >>> Aljoscha did the reduce. I will do CoGroup.
> > >>>
> > >>
> > >>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message