flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From mnxfst <mnx...@gmail.com>
Subject Re: On some GUI tools for building Flink Streaming data flows...
Date Tue, 11 Aug 2015 10:04:11 GMT
Hi Gyula, hi Slim,

being the core developer of SPQR, I am very glad to see the framework
referenced
on the Apache Flink dev list ;-)

Following the same approach as Flink - using real data streaming instead of
mini-batching - SPQR
adds some ad hoc'ness features to introduce more flexibility into the
streaming model. The
core idea is to shield away complex topology implementation and deployments
but provide 
a more business oriented layer for ad hoc queries and ad hoc stream
modificiation.

>From our point of view both features prepare stream processing for usage
within contexts that
are less tech oriented and where SQL is a broadly spoken language. But .. it
is not about 
refactoring SQL to stream processing (as I am not sure if a language
specified for static data is
suitable for streaming data) but to find a suitable approach which is
somehow similar to SQL
when it comes to adopting it in a business oriented context.

The most accurate metaphor for describing how we expect stream processing to
look like
for its users is that of a plumber working on a (more or less) large pipe
system. 

As both approaches are quite similar but Flink already provides some very
ice features that SPQR 
lacks, eg graph optimization & distribution and cluster support, I think it
may be worth checking if 
porting our vision of ad hoc'ness into Flink is possible - maybe as a layer
on top of the  
framework or as more core integration. Therefore - if such a feature is of
any interest 
for you - I would like to offer support on that task.

As outlined above and as Slim also mentioned it regarding StreamFlow UI
providing a less tech oriented
but more business oriented layer may lay ground for a broad adoption of
Flink since NONE of the
existing frameworks provides such a feature. All the other kids on the block
require heavy code
compilation and deployment before any work is done.

Cheers,
  Christian (@mnxfst)



--
View this message in context: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/On-some-GUI-tools-for-building-Flink-Streaming-data-flows-tp7233p7453.html
Sent from the Apache Flink Mailing List archive. mailing list archive at Nabble.com.

Mime
View raw message