flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From godfrey he <godfre...@gmail.com>
Subject Re: Flink SQL Gateway
Date Fri, 17 Apr 2020 02:37:52 GMT
Hi Flavio,

Thanks for the detailed explanation.

I think we should let Catalog know this concept first,
then TableEnvironment or SQL Gateway can do more stuff based on that.
But "trigger" is Database domain concept, I think it need more discuss
whether Flink should support this. also cc @bowenli86@gmail.com

Best,
Godfrey

Jeff Zhang <zjffdu@gmail.com> 于2020年4月16日周四 下午10:54写道:

> Hi Flavio,
>
> If you would like to use have a UI to register data sources, run flink sql
> and preview the sql result, then you can use zeppelin directly. You can
> check the tutorial here,
> 1) Get started https://link.medium.com/oppqD6dIg5
> <https://t.co/PTouUYYTrv?amp=1> 2) Batch https://
> link.medium.com/3qumbwRIg5 <https://t.co/Yo9QAY0Joj?amp=1> 3) Streaming
> https://link.medium.com/RBHa2lTIg5 <https://t.co/sUapN40tvI?amp=1> 4)
> Advanced usage https://link.medium.com/CAekyoXIg5
> <https://t.co/MXolULmafZ?amp=1>
>
> And here's one article shared by someone else about how to use flink on
> zeppelin.
>
> https://medium.com/@abdelkrim.hadjidj/event-driven-supply-chain-for-crisis-with-flinksql-be80cb3ad4f9
>
> Besides that, Zeppelin provides rest api which you can use to integarte
> with other system, but it is not standard jdbc protocol.
> http://zeppelin.apache.org/docs/0.9.0-preview1/usage/rest_api/notebook.html
>
> And I am doing more improvement recently, I will reveal more details in
> next week's flink forward.
>
> https://www.flink-forward.org/sf-2020/conference-program#it%E2%80%99s-finally-here--python-on-flink---flink-on-zeppelin
>
>
>
>
>
> Flavio Pompermaier <pompermaier@okkam.it> 于2020年4月16日周四 下午8:24写道:
>
>> Basically we want to give a UI to the user to register its data sources
>> (i.e. catalogs in the Flink world), preview them (SELECT * LIMIT 100 for
>> example) but, in the case of JDBC catalogs, also to see relationships and
>> triggers.
>> We don't want to reimplement the wheel so we would like to reuse and
>> contribute to Flink as much as possible (since then in the batch jobs we
>> use Flink and we don't like to do the same work twice..).
>> In this way we can contribute to Flink if something is missing in the SQL
>> Gateway. However I don't know how to extend the existing stuff (for example
>> if I want table relationships and triggers)..
>>
>> Best,
>> Flavio
>>
>> On Thu, Apr 16, 2020 at 1:38 PM godfrey he <godfreyhe@gmail.com> wrote:
>>
>>> Hi Flavio,
>>>
>>> Since 1.11(master), Flink supports "CREATE CATALOG ..." [1],
>>> we can use this statement create catalog dynamically.
>>>
>>> Currently, Catalog[2] dose not supports any operations on TRIGGER.
>>> Flink can't also use such info now. What's your user scenario?
>>>
>>> [1] https://issues.apache.org/jira/browse/FLINK-15349
>>> [2]
>>> https://ci.apache.org/projects/flink/flink-docs-stable/dev/table/catalogs.html#catalog-api
>>>
>>> Best,
>>> Godfrey
>>>
>>> Flavio Pompermaier <pompermaier@okkam.it> 于2020年4月16日周四 下午6:16写道:
>>>
>>>> Hi Godfrey,
>>>> I'd like to use the SQL gateway as a data proxy in our architecture.
>>>> However, catalogs in our use case are not know at configuration time..
>>>> is there a way to permit to register a JDBC catalog (for example when I
>>>> want to connect to a Postgres database)?
>>>> What if I want to add SHOW TRIGGERS? Do you think it could be
>>>> interesting?
>>>>
>>>> On Thu, Apr 16, 2020 at 10:55 AM godfrey he <godfreyhe@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi Flavio,
>>>>>
>>>>> We prose FLIP-91[1] to support SQL Gateway at the beginning of this
>>>>> year.
>>>>> After a long discussion, we reached an agreement that
>>>>> SQL Gateway is an eco-system under ververia as first step.[2]
>>>>> Which could help SQL Gateway move forward faster.
>>>>> Now we almost finish first version development, some users are trying
>>>>> it out.
>>>>> Any suggestions are welcome!
>>>>>
>>>>> [1]
>>>>> https://docs.google.com/document/d/1DKpFdov1o_ObvrCmU-5xi-VrT6nR2gxq-BbswSSI9j8/edit?ts=5e4cd711#heading=h.sru3xzbt2i1k
>>>>> [2] https://github.com/ververica/flink-sql-gateway
>>>>>
>>>>> Best,
>>>>> Godfrey
>>>>>
>>>>> Flavio Pompermaier <pompermaier@okkam.it> 于2020年4月16日周四
下午4:42写道:
>>>>>
>>>>>> Hi Jeff,
>>>>>> FLIP-24 [1] proposed to develop a SQL gateway to query Flink via
SQL
>>>>>> but since then no progress has been made on that point. Do you think
that
>>>>>> Zeppelin could be used somehow as a SQL Gateway towards Flink for
the
>>>>>> moment?
>>>>>> Any chance that a Flink SQL Gateway could ever be developed? Is there
>>>>>> anybody interested in this?
>>>>>>
>>>>>> Best,
>>>>>> Flavio
>>>>>>
>>>>>> [1]
>>>>>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-24+-+SQL+Client
>>>>>>
>>>>>
>
> --
> Best Regards
>
> Jeff Zhang
>

Mime
View raw message