flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Becket Qin <becket....@gmail.com>
Subject Re: [PROPOSAL] Contribute Stateful Functions to Apache Flink
Date Mon, 14 Oct 2019 08:05:27 GMT
+1 to adding Stateful Function to Flink. It is a very useful addition to
the Flink ecosystem.

Given this is essentially a new top-level / first-citizen API of Flink, it
seems better to have it the Flink core repo. This will also avoid letting
this important new API to be blocked on potential problems of maintaining
multiple different repositories.

Thanks,

Jiangjie (Becket) Qin

On Sun, Oct 13, 2019 at 4:48 AM Hequn Cheng <chenghequn@gmail.com> wrote:

> Hi Stephan,
>
> Big +1 for adding this to Apache Flink!
>
> As for the problem of whether this should be added to the Flink main
> repository, from my side, I prefer to put it in the main repository. Not
> only Stateful Functions shares very close relations with the current Flink,
> but also other libs or modules in Flink can make use of it the other way
> round in the future. At that time the Flink API stack would also be changed
> a bit and this would be cool.
>
> Best, Hequn
>
> On Sat, Oct 12, 2019 at 9:16 PM Biao Liu <mmyy1110@gmail.com> wrote:
>
>> Hi Stehpan,
>>
>> +1 for having Stateful Functions in Flink.
>>
>> Before discussing which repository it should belong, I was wondering if
>> we have reached an agreement of "splitting flink repository" as Piotr
>> mentioned or not. It seems that it's just no more further discussion.
>> It's OK for me to add it to core repository. After all almost everything
>> is in core repository now. But if we decide to split the core repository
>> someday, I tend to create a separate repository for Stateful Functions. It
>> might be good time to take the first step of splitting.
>>
>> Thanks,
>> Biao /'bɪ.aʊ/
>>
>>
>>
>> On Sat, 12 Oct 2019 at 19:31, Yu Li <carp84@gmail.com> wrote:
>>
>>> Hi Stephan,
>>>
>>> Big +1 for adding stateful functions to Flink. I believe a lot of user
>>> would be interested to try this out and I could imagine how this could
>>> contribute to reduce the TCO for business requiring both streaming
>>> processing and stateful functions.
>>>
>>> And my 2 cents is to put it into flink core repository since I could see
>>> a tight connection between this library and flink state.
>>>
>>> Best Regards,
>>> Yu
>>>
>>>
>>> On Sat, 12 Oct 2019 at 17:31, jincheng sun <sunjincheng121@gmail.com>
>>> wrote:
>>>
>>>> Hi Stephan,
>>>>
>>>> bit +1 for adding this great features to Apache Flink.
>>>>
>>>> Regarding where we should place it, put it into Flink core repository
>>>> or create a separate repository? I prefer put it into main repository and
>>>> looking forward the more detail discussion for this decision.
>>>>
>>>> Best,
>>>> Jincheng
>>>>
>>>>
>>>> Jingsong Li <jingsonglee0@gmail.com> 于2019年10月12日周六 上午11:32写道:
>>>>
>>>>> Hi Stephan,
>>>>>
>>>>> big +1 for this contribution. It provides another user interface that
>>>>> is easy to use and popular at this time. these functions, It's hard for
>>>>> users to write in SQL/TableApi, while using DataStream is too complex.
>>>>> (We've done some stateFun kind jobs using DataStream before). With
>>>>> statefun, it is very easy.
>>>>>
>>>>> I think it's also a good opportunity to exercise Flink's core
>>>>> capabilities. I looked at stateful-functions-flink briefly, it is very
>>>>> interesting. I think there are many other things Flink can improve. So
I
>>>>> think it's a better thing to put it into Flink, and the improvement for
it
>>>>> will be more natural in the future.
>>>>>
>>>>> Best,
>>>>> Jingsong Lee
>>>>>
>>>>> On Fri, Oct 11, 2019 at 7:33 PM Dawid Wysakowicz <
>>>>> dwysakowicz@apache.org> wrote:
>>>>>
>>>>>> Hi Stephan,
>>>>>>
>>>>>> I think this is a nice library, but what I like more about it is
that
>>>>>> it suggests exploring different use-cases. I think it definitely
makes
>>>>>> sense for the Flink community to explore more lightweight applications
that
>>>>>> reuses resources. Therefore I definitely think it is a good idea
for Flink
>>>>>> community to accept this contribution and help maintaining it.
>>>>>>
>>>>>> Personally I'd prefer to have it in a separate repository. There
were
>>>>>> a few discussions before where different people were suggesting to
extract
>>>>>> connectors and other libraries to separate repositories. Moreover
I think
>>>>>> it could serve as an example for the Flink ecosystem website[1].
This could
>>>>>> be the first project in there and give a good impression that the
community
>>>>>> sees potential in the ecosystem website.
>>>>>>
>>>>>> Lastly, I'm wondering if this should go through PMC vote according
to
>>>>>> our bylaws[2]. In the end the suggestion is to adopt an existing
code base
>>>>>> as is. It also proposes a new programs concept that could result
in a shift
>>>>>> of priorities for the community in a long run.
>>>>>>
>>>>>> Best,
>>>>>>
>>>>>> Dawid
>>>>>>
>>>>>> [1]
>>>>>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Create-a-Flink-ecosystem-website-td27519.html
>>>>>>
>>>>>> [2] https://cwiki.apache.org/confluence/display/FLINK/Flink+Bylaws
>>>>>> On 11/10/2019 13:12, Till Rohrmann wrote:
>>>>>>
>>>>>> Hi Stephan,
>>>>>>
>>>>>> +1 for adding stateful functions to Flink. I believe the new set
of
>>>>>> applications this feature will unlock will be super interesting for
new and
>>>>>> existing Flink users alike.
>>>>>>
>>>>>> One reason for not including it in the main repository would to not
>>>>>> being bound to Flink's release cadence. This would allow to release
faster
>>>>>> and more often. However, I believe that having it eventually in Flink's
>>>>>> main repository would be beneficial in the long run.
>>>>>>
>>>>>> Cheers,
>>>>>> Till
>>>>>>
>>>>>> On Fri, Oct 11, 2019 at 12:56 PM Trevor Grant <
>>>>>> trevor.d.grant@gmail.com> wrote:
>>>>>>
>>>>>>> +1 non-binding on contribution.
>>>>>>>
>>>>>>> Separate repo, or feature branch to start maybe? I just feel
like in
>>>>>>> the beginning this thing is going to have lots of breaking changes
that
>>>>>>> maybe aren't going to fit well with tests / other "v1+" release
code. Just
>>>>>>> my .02.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Oct 11, 2019 at 4:38 AM Stephan Ewen <sewen@apache.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Dear Flink Community!
>>>>>>>>
>>>>>>>> Some of you probably heard it already: On Tuesday, at Flink
Forward
>>>>>>>> Berlin, we announced **Stateful Functions**.
>>>>>>>>
>>>>>>>> Stateful Functions is a library on Flink to implement general
>>>>>>>> purpose applications. It is built around stateful functions
(who would have
>>>>>>>> thunk)
>>>>>>>> that can communicate arbitrarily through messages, have consistent
>>>>>>>> state, and a small resource footprint. They are a bit like
keyed
>>>>>>>> ProcessFunctions
>>>>>>>> that can send each other messages.
>>>>>>>> As simple as this sounds, this means you can now communicate
in
>>>>>>>> non-DAG patterns, so it allows users to build programs they
cannot build
>>>>>>>> with Flink.
>>>>>>>> It also has other neat properties, like multiplexing of functions,
>>>>>>>> modular composition, tooling both container-based deployments
and
>>>>>>>> as-a-Flink-job deployments.
>>>>>>>>
>>>>>>>> You can find out more about it here
>>>>>>>>   - Website: https://statefun.io/
>>>>>>>>   - Code: https://github.com/ververica/stateful-functions
>>>>>>>>   - Talk with motivation:
>>>>>>>> https://speakerdeck.com/stephanewen/stateful-functions-building-general-purpose-applications-and-services-on-apache-flink?slide=12
>>>>>>>>
>>>>>>>>
>>>>>>>> Now for the main issue: **We would like to contribute this
project
>>>>>>>> to Apache Flink**
>>>>>>>>
>>>>>>>> I believe that this is a great fit for both sides.
>>>>>>>> For the Flink community, it would be a way to extend the
>>>>>>>> capabilities and use cases of Flink into a completely different
type of
>>>>>>>> applications and thus grow the community into this new field.
>>>>>>>> Many discussions recently about evolving the Flink runtime
(both on
>>>>>>>> the mailing list and at conferences) show the interest in
Flink users in
>>>>>>>> the space that Stateful Functions covers.
>>>>>>>> It seems natural that Stateful Functions should closely co-develop
>>>>>>>> with Apache Flink, ideally as part of the project.
>>>>>>>>
>>>>>>>> There are many details to be discusses, for example whether
this
>>>>>>>> should be added to the Flink core repository, or whether
we and to create a
>>>>>>>> separate repository
>>>>>>>> for this. But I think we should start discussing this after
we have
>>>>>>>> consensus on whether the community wants this contribution.
>>>>>>>>
>>>>>>>> Really looking forward to hear what you think!
>>>>>>>>
>>>>>>>> Best Regards,
>>>>>>>> Stephan
>>>>>>>>
>>>>>>>>
>>>>>
>>>>> --
>>>>> Best, Jingsong Lee
>>>>>
>>>>

Mime
View raw message