flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chiwan Park <chiwanp...@apache.org>
Subject Re: Effort to add SQL / StreamSQL to Flink
Date Mon, 11 Jan 2016 02:31:28 GMT
We still don’t have a concensus about the streaming SQL and CEP library on Flink. Some people
want to merge these two libraries. Maybe we have to discuss about this in mailing list.

> On Jan 11, 2016, at 10:53 AM, Nick Dimiduk <ndimiduk@gmail.com> wrote:
> 
> What's the relationship between the streaming SQL proposed here and the CEP
> syntax proposed earlier in the week?
> 
> On Sunday, January 10, 2016, Henry Saputra <henry.saputra@gmail.com> wrote:
> 
>> Awesome! Thanks for the reply, Fabian.
>> 
>> - Henry
>> 
>> On Sunday, January 10, 2016, Fabian Hueske <fhueske@gmail.com
>> <javascript:;>> wrote:
>> 
>>> Hi Henry,
>>> 
>>> There is https://issues.apache.org/jira/browse/FLINK-2099 and a few
>>> subissues.
>>> I'll reorganize these and add more issues for the tasks described in the
>>> design document in the next days.
>>> 
>>> Thanks, Fabian
>>> 
>>> 2016-01-10 2:45 GMT+01:00 Henry Saputra <henry.saputra@gmail.com
>> <javascript:;>
>>> <javascript:;>>:
>>> 
>>>> HI Fabian,
>>>> 
>>>> Have you created JIRA ticket to keep track of this new feature?
>>>> 
>>>> - Henry
>>>> 
>>>> On Thu, Jan 7, 2016 at 6:05 AM, Fabian Hueske <fhueske@gmail.com
>> <javascript:;>
>>> <javascript:;>> wrote:
>>>>> Hi everybody,
>>>>> 
>>>>> in the last days, Timo and I refined the design document for adding a
>>>> SQL /
>>>>> StreamSQL interface on top of Flink that was started by Stephan.
>>>>> 
>>>>> The document proposes an architecture that is centered around Apache
>>>>> Calcite. Calcite is an Apache top-level project and includes a SQL
>>>> parser,
>>>>> a semantic validator for relational queries, and a rule- and
>> cost-based
>>>>> relational optimizer. Calcite is used by Apache Hive and Apache Drill
>>>>> (among other projects). In a nutshell, the plan is to translate Table
>>> API
>>>>> and SQL queries into Calcite's relational expression trees, optimize
>>>> these
>>>>> trees, and translate them into DataSet and DataStream programs.The
>>>> document
>>>>> breaks down the work into several tasks and subtasks.
>>>>> 
>>>>> Please review the design document and comment.
>>>>> 
>>>>> -- >
>>>>> 
>>>> 
>>> 
>> https://docs.google.com/document/d/1TLayJNOTBle_-m1rQfgA6Ouj1oYsfqRjPcp1h2TVqdI/edit?usp=sharing
>>>>> 
>>>>> Unless there are major concerns with the design, Timo and I want to
>>> start
>>>>> next week to move the current Table API on top of Apache Calcite
>> (Task
>>> 1
>>>> in
>>>>> the document). The goal of this task is to have the same
>> functionality
>>> as
>>>>> currently, but with Calcite in the translation process. This is a
>>>> blocking
>>>>> task that we hope to complete soon. Afterwards, we can independently
>>> work
>>>>> on different aspects such as extending the Table API, adding a SQL
>>>>> interface (basically just a parser), integration with external data
>>>>> sources, better code generation, optimization rules, streaming
>> support
>>>> for
>>>>> the Table API, StreamSQL, etc..
>>>>> 
>>>>> Timo and I plan to work on a WIP branch to implement Task 1 and merge
>>> it
>>>> to
>>>>> the master branch once the task is completed. Of course, everybody is
>>>>> welcome to contribute to this effort. Please let us know such that we
>>> can
>>>>> coordinate our efforts.
>>>>> 
>>>>> Thanks,
>>>>> Fabian

Regards,
Chiwan Park



Mime
View raw message