flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Márton Balassi <mbala...@apache.org>
Subject Re: Basic lambda prototype :)
Date Sat, 27 Sep 2014 06:00:06 GMT
Wow, that is in fact quite impressive. :)

On Fri, Sep 26, 2014 at 11:21 PM, Gyula Fóra <gyfora@apache.org> wrote:

> Hey guys,
>
> I am trying contain my excitement, but I am failing :)
>
> So today I implemented a basic prototype for connecting batch processing
> and streaming jobs as a good starting point for a 'lambda architecture
> api'. And most surprisingly it works :) It is very simple so far (currently
> only works for map operator and chaining is turned off) and some hacking
> along the way, but I hope you will like it.
>
> You can see the commit here:
>
> https://github.com/mbalassi/incubator-flink/commit/7bfc16a7803e1f2cc4b38c746af069aab4450637
>
> And a sample job:
>
> https://github.com/mbalassi/incubator-flink/blob/7bfc16a7803e1f2cc4b38c746af069aab4450637/flink-addons/flink-streaming/flink-streaming-core/src/test/java/org/apache/flink/streaming/api/LambdaTest.java
>
> And for the others, it looks something like this:
>
>  ExecutionEnvironment batchEnv = ExecutionEnvironment.
> *createLocalEnvironment*(1);
> StreamExecutionEnvironment streamEnv = StreamExecutionEnvironment.
> *createLocalEnvironment*(1);
>
> DataSet<Long> dataSet = batchEnv.generateSequence(1, 10).map(*new*
> Increment());
>
> DataStream<Long> dataStream = streamEnv.generateSequence(10, 20);
>
>   *dataStream.lambdaJoin(dataSet).map(new Increment()).print();*
> dataSet.print();
>
>  * LambdaEnvironment lambdaEnv = new LambdaEnvironment(batchEnv,
> streamEnv);*
>
> * lambdaEnv.execute(":)");*
>
>
> I hope I could get you excited too :)
>
> Cheers,
>
> Gyula
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message