flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gyula Fóra <gyf...@apache.org>
Subject Basic lambda prototype :)
Date Fri, 26 Sep 2014 21:21:43 GMT
Hey guys,

I am trying contain my excitement, but I am failing :)

So today I implemented a basic prototype for connecting batch processing
and streaming jobs as a good starting point for a 'lambda architecture
api'. And most surprisingly it works :) It is very simple so far (currently
only works for map operator and chaining is turned off) and some hacking
along the way, but I hope you will like it.

You can see the commit here:

And a sample job:

And for the others, it looks something like this:

 ExecutionEnvironment batchEnv = ExecutionEnvironment.
StreamExecutionEnvironment streamEnv = StreamExecutionEnvironment.

DataSet<Long> dataSet = batchEnv.generateSequence(1, 10).map(*new*

DataStream<Long> dataStream = streamEnv.generateSequence(10, 20);

  *dataStream.lambdaJoin(dataSet).map(new Increment()).print();*

 * LambdaEnvironment lambdaEnv = new LambdaEnvironment(batchEnv,

* lambdaEnv.execute(":)");*

I hope I could get you excited too :)



  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message