flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Flavio Pompermaier <pomperma...@okkam.it>
Subject Re: Flink program without a line of code
Date Sat, 23 Apr 2016 07:09:32 GMT
Hi Alexander,
since I was looking for something similar some days ago here is what I know
about this argument:
during the Stratosphere project there was Meteor and Supremo allowing that
[1] but then it was dismissed in favour of Pig integration that I don't
wheter it was ever completed. Yiu could give a try to Piglet project[2]
that allows to use PIG with Spark and Flink but I don't know how well it
works (Flink integration is also very recent and not documented anywhere).


[1] http://stratosphere.eu/assets/papers/Sopremo_Meteor%20BigData.pdf
[2] https://github.com/ksattler/piglet
On 23 Apr 2016 07:48, "Aljoscha Krettek" <aljoscha@apache.org> wrote:

I think if the Table API/SQL API evolves enough it should be able to supply
a Flink program as just an SQL query with source/sink definitions.
Hopefully, in the future. :-)


On Fri, 22 Apr 2016 at 23:10 Fabian Hueske <fhueske@gmail.com> wrote:

> Hi Alex,
> welcome to the Flink community!
> Right now, there is no way to specify a Flink program without writing code
> (Java, Scala, Python(beta)).
> In principle it is possible to put such functionality on top of the
> DataStream or DataSet APIs.
> This has been done before for other programming APIs (Flink's own
> libraries Table API, Gelly, FlinkML, and externals Apache Beam / Google
> DataFlow, Mahout, Cascading, ...). However, all of these are again
> programming APIs, some specialized for certain use-cases.
> Specifying Flink programs by config files (or graphically) would require a
> data model, a DataStream/DataSet program generator and probably a code
> generation component.
> Best, Fabian
> 2016-04-22 18:41 GMT+02:00 Alexander Smirnov <alexander.smirnoff@gmail.com
> >:
>> Hi guys!
>> I’m new to Flink, and actually to this mailing list as well :) this is my
>> first message.
>> I’m still reading the documentation and I would say Flink is an amazing
>> system!! Thanks everybody who participated in the development!
>> The information I didn’t find in the documentation - if it is possible to
>> describe data(stream) transformation without any code (Java/Scala).
>> I mean if it is possible to describe datasource functions, all of the
>> operators, connections between them, and sinks in a plain text
>> configuration file and then feed it to Flink.
>> In this case it would be possible to change data flow without
>> recompilation/redeployment.
>> Is there a similar functionality in Flink? May be some third party plugin?
>> Thank you,
>> Alex

View raw message