From user-return-30291-archive-asf-public=cust-asf.ponee.io@flink.apache.org Fri Oct 11 11:26:04 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 9B4A0180656 for ; Fri, 11 Oct 2019 13:26:04 +0200 (CEST) Received: (qmail 54476 invoked by uid 500); 11 Oct 2019 11:25:57 -0000 Mailing-List: contact user-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list user@flink.apache.org Received: (qmail 54454 invoked by uid 99); 11 Oct 2019 11:25:56 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 11 Oct 2019 11:25:56 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 33E591A44DC; Fri, 11 Oct 2019 11:25:56 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 0.001 X-Spam-Level: X-Spam-Status: No, score=0.001 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.2, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-he-de.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id FLrEf5um3u-a; Fri, 11 Oct 2019 11:25:54 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=2607:f8b0:4864:20::842; helo=mail-qt1-x842.google.com; envelope-from=imjark@gmail.com; receiver= Received: from mail-qt1-x842.google.com (mail-qt1-x842.google.com [IPv6:2607:f8b0:4864:20::842]) by mx1-he-de.apache.org (ASF Mail Server at mx1-he-de.apache.org) with ESMTPS id B96517F589; Fri, 11 Oct 2019 11:19:37 +0000 (UTC) Received: by mail-qt1-x842.google.com with SMTP id v52so13277014qtb.8; Fri, 11 Oct 2019 04:19:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=ZFWgZOibOxchgIsQpqWRW43qreY/Y3ws0+82eFT5vy0=; b=gY6opGDBldNO5zcxacwnB60B8x4w2QTmiFjG/Y8om/XBd1otN/3noFJXoqVy9+v3mN 80VSR/sa0JJL4/t7txtLhPKp5+tetS1OsViynT5Pr5AjwtyPP4BegBxW4hVDj+Hq87Ci 41fS6th+1wfLX5F+PYVB4ElgRo7GC+kBpDvixjp8CBpuP6cmQ/7wusjFStAptHpuF50V hBRFRyzDaZ2ILg4ifKXw4pEne7pg2B8sdyuconiJheLVjFUkDTzn2KObkh/TkUdhBvnZ qqExpNwQqL8uRJCxl6+pZX/Fu0dy08Th+YhJyVL/0JmLWPGlY3HqXuFZhOrDqaVLo5o/ JYbQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=ZFWgZOibOxchgIsQpqWRW43qreY/Y3ws0+82eFT5vy0=; b=eJyuVqHgpwVlk/BVz07A+eTB3pD0xrP3iYJHgiuGjnFD01obzN+ICAIUWVD/BOhRS8 wwgTBcAX7EwpI44jMSuLtkLGOJ11mgYJtWTj7tAB6wGI4lJn+Jsrg/grsUjwBgNCZ6IY bk+UQipqihzZwUeanLbZORNYk/DuMESuEyaZap13bZIjvxhjn/LiO946aU73uxKB6AY3 8au2YYmm7IDqZHnT919zt/1TZ1Zb1Ic2mmpW/9YrXUIqlAh8JxzqlNVXk2mfK+R05q7D LGdJ8pmycvduObolhEuUFh2grrYi4X5Ff/+kgvojEwHJG5PFW7Fx6191+v1QvPDN+rkV r/VQ== X-Gm-Message-State: APjAAAVOzbl8cjwqkSvI/2VdC6+Z10V/gcnPpWKjluRRazHiSGzdsfpb GUVW5IKoaSzs2Q+OyMM8imQfYx2paOm5qP00pfU= X-Google-Smtp-Source: APXvYqwZW41cf3l5aO7H8sMS3PAqEcY8DVMxGgfsLeu7+tE6G9I4MCfwD8PW796o91H5gInQo7cd6cSDg10ot7MeeXU= X-Received: by 2002:a0c:ec46:: with SMTP id n6mr15508902qvq.23.1570792771325; Fri, 11 Oct 2019 04:19:31 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Jark Wu Date: Fri, 11 Oct 2019 19:19:20 +0800 Message-ID: Subject: Re: [PROPOSAL] Contribute Stateful Functions to Apache Flink To: Till Rohrmann Cc: Trevor Grant , Stephan Ewen , dev , user Content-Type: multipart/alternative; boundary="00000000000026affc0594a0b17d" --00000000000026affc0594a0b17d Content-Type: text/plain; charset="UTF-8" Hi Stephan, big +1 for adding statefun to Apache Flink. I think we can add it in the main repository as a library just like State Process API. Having it in the main repository can closely co-develop with Apache Flink which be beneficial for both side. Regards, Jark On Fri, 11 Oct 2019 at 19:12, Till Rohrmann wrote: > Hi Stephan, > > +1 for adding stateful functions to Flink. I believe the new set of > applications this feature will unlock will be super interesting for new and > existing Flink users alike. > > One reason for not including it in the main repository would to not being > bound to Flink's release cadence. This would allow to release faster and > more often. However, I believe that having it eventually in Flink's main > repository would be beneficial in the long run. > > Cheers, > Till > > On Fri, Oct 11, 2019 at 12:56 PM Trevor Grant > wrote: > >> +1 non-binding on contribution. >> >> Separate repo, or feature branch to start maybe? I just feel like in the >> beginning this thing is going to have lots of breaking changes that maybe >> aren't going to fit well with tests / other "v1+" release code. Just my >> .02. >> >> >> >> On Fri, Oct 11, 2019 at 4:38 AM Stephan Ewen wrote: >> >>> Dear Flink Community! >>> >>> Some of you probably heard it already: On Tuesday, at Flink Forward >>> Berlin, we announced **Stateful Functions**. >>> >>> Stateful Functions is a library on Flink to implement general purpose >>> applications. It is built around stateful functions (who would have thunk) >>> that can communicate arbitrarily through messages, have consistent >>> state, and a small resource footprint. They are a bit like keyed >>> ProcessFunctions >>> that can send each other messages. >>> As simple as this sounds, this means you can now communicate in non-DAG >>> patterns, so it allows users to build programs they cannot build with Flink. >>> It also has other neat properties, like multiplexing of functions, >>> modular composition, tooling both container-based deployments and >>> as-a-Flink-job deployments. >>> >>> You can find out more about it here >>> - Website: https://statefun.io/ >>> - Code: https://github.com/ververica/stateful-functions >>> - Talk with motivation: >>> https://speakerdeck.com/stephanewen/stateful-functions-building-general-purpose-applications-and-services-on-apache-flink?slide=12 >>> >>> >>> Now for the main issue: **We would like to contribute this project to >>> Apache Flink** >>> >>> I believe that this is a great fit for both sides. >>> For the Flink community, it would be a way to extend the capabilities >>> and use cases of Flink into a completely different type of applications and >>> thus grow the community into this new field. >>> Many discussions recently about evolving the Flink runtime (both on the >>> mailing list and at conferences) show the interest in Flink users in the >>> space that Stateful Functions covers. >>> It seems natural that Stateful Functions should closely co-develop with >>> Apache Flink, ideally as part of the project. >>> >>> There are many details to be discusses, for example whether this should >>> be added to the Flink core repository, or whether we and to create a >>> separate repository >>> for this. But I think we should start discussing this after we have >>> consensus on whether the community wants this contribution. >>> >>> Really looking forward to hear what you think! >>> >>> Best Regards, >>> Stephan >>> >>> --00000000000026affc0594a0b17d Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi Stephan,

big=C2=A0+1 for addin= g statefun to Apache Flink.=C2=A0

I think we can add it = in the main repository as a library just like State Process API.=C2=A0
Having it in the main repository can closely co-develop with Apache F= link which be beneficial for both side.

Regards,
Jark


On Fri, 11 Oct 2019 at 19:= 12, Till Rohrmann <trohrmann@apa= che.org> wrote:
Hi Stephan,

+1 for adding statef= ul functions to Flink. I believe the new set of applications this feature w= ill unlock will be super interesting for new and existing Flink users alike= .

One reason for not including it in the main = repository would to not being bound to Flink's release cadence. This wo= uld allow to release faster and more often. However, I believe that having = it eventually in Flink's main repository would be beneficial in the lon= g run.

Cheers,
Till

On Fri, Oct 11,= 2019 at 12:56 PM Trevor Grant <trevor.d.grant@gmail.com> wrote:
+1 non-bindi= ng on contribution.=C2=A0

Separate=C2=A0repo, or fea= ture branch to start maybe? I just feel like in the beginning this thing is= going to have lots of breaking changes that maybe aren't going to fit = well with tests / other=C2=A0"v1+" release code. Just my .02.=C2= =A0



On Fri, Oct 11, 2019 at 4:38 AM Step= han Ewen <sewen@ap= ache.org> wrote:
Dear Flink Community!

Some of y= ou probably heard it already: On Tuesday, at Flink Forward Berlin, we annou= nced **Stateful Functions**.

Stateful Functions is= a library on Flink to implement general purpose applications. It is built = around stateful functions (who would have thunk)
that can communi= cate arbitrarily through messages, have consistent state, and a small resou= rce footprint. They are a bit like keyed ProcessFunctions
that ca= n send each other messages.
As simple as this sounds, this means = you can now communicate in non-DAG patterns, so it allows users to build pr= ograms they cannot build with Flink.
It also has other neat prope= rties, like multiplexing of functions, modular composition, tooling=C2=A0bo= th container-based deployments and as-a-Flink-job deployments.
You can find out more about it here
=C2=A0 - Website= :=C2=A0https://statefun.= io/


Now for the main issue: **We would like to= contribute this project to Apache Flink**

I belie= ve that this is a great fit for both sides.
For the Flink communi= ty, it would be a way to extend the capabilities and use cases of Flink int= o a completely different type of applications and thus grow the community i= nto this new field.
Many discussions recently about evolving the = Flink runtime (both on the mailing list and at conferences) show the intere= st in Flink users in the space that Stateful Functions covers.
It= seems natural that Stateful Functions should closely co-develop with Apach= e Flink, ideally as part of the project.

There are= many details to be discusses, for example whether this should be added to = the Flink core repository, or whether we and to create a separate repositor= y
for this. But I think we should start discussing this after we = have consensus on whether the community wants this contribution.
=
Really looking forward to hear what you think!
Best Regards,
Stephan

--00000000000026affc0594a0b17d--