flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shaoxuan Wang <wshaox...@gmail.com>
Subject [DISCUSS] FLIP-39: Flink ML pipeline and ML libs
Date Mon, 29 Apr 2019 02:34:22 GMT
Hi everyone,

Weihua has proposed to rebuild Flink ML pipeline on top of TableAPI several
months ago in this mail thread:

http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Embracing-Table-API-in-Flink-ML-td25368.html

Luogen, Becket, Xu, Weihua and I have been working on this proposal offline in
the past a few months. Now we want to share the first phase of the entire
proposal with a FLIP. In this FLIP-39, we want to achieve several things
(and hope those can be accomplished and released in Flink-1.9):

   -

   Provide a new set of ML core interface (on top of Flink TableAPI)
   -

   Provide a ML pipeline interface (on top of Flink TableAPI)
   -

   Provide the interfaces for parameters management and pipeline/mode
   persistence
   -

   All the above interfaces should facilitate any new ML algorithm. We will
   gradually add various standard ML algorithms on top of these new proposed
   interfaces to ensure their feasibility and scalability.


Part of this FLIP has been present in Flink Forward 2019 @ San Francisco by
Xu and Me.

https://sf-2019.flink-forward.org/conference-program#when-table-meets-ai--build-flink-ai-ecosystem-on-table-api

https://sf-2019.flink-forward.org/conference-program#high-performance-ml-library-based-on-flink

You can find the videos & slides at
https://www.ververica.com/flink-forward-san-francisco-2019

The design document for FLIP-39 can be found here:

https://docs.google.com/document/d/1StObo1DLp8iiy0rbukx8kwAJb0BwDZrQrMWub3DzsEo


I am looking forward to your feedback.

Regards,

Shaoxuan

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message