flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shaoxuan Wang <wshaox...@gmail.com>
Subject [DISCUSS] FLIP-39: Flink ML pipeline and ML libs
Date Mon, 29 Apr 2019 02:34:22 GMT
Hi everyone,

Weihua has proposed to rebuild Flink ML pipeline on top of TableAPI several
months ago in this mail thread:


Luogen, Becket, Xu, Weihua and I have been working on this proposal offline in
the past a few months. Now we want to share the first phase of the entire
proposal with a FLIP. In this FLIP-39, we want to achieve several things
(and hope those can be accomplished and released in Flink-1.9):


   Provide a new set of ML core interface (on top of Flink TableAPI)

   Provide a ML pipeline interface (on top of Flink TableAPI)

   Provide the interfaces for parameters management and pipeline/mode

   All the above interfaces should facilitate any new ML algorithm. We will
   gradually add various standard ML algorithms on top of these new proposed
   interfaces to ensure their feasibility and scalability.

Part of this FLIP has been present in Flink Forward 2019 @ San Francisco by
Xu and Me.



You can find the videos & slides at

The design document for FLIP-39 can be found here:


I am looking forward to your feedback.



  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message