Mailing-List: contact dev-help@flink.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@flink.apache.org
MIME-Version: 1.0
In-Reply-To: <CACTd3c90-CUMF+Nx9PF5PTBTq1ng4-BjxFrDsZnmjNM-GUJH-Q@mail.gmail.com>
References: <CACTd3c90-CUMF+Nx9PF5PTBTq1ng4-BjxFrDsZnmjNM-GUJH-Q@mail.gmail.com>
From: Fabian Hueske <fhueske@gmail.com>
Date: Mon, 27 Nov 2017 15:23:40 +0100
Message-ID: <CAAdrtT19UTUJz1fMD1r-3qybmdRhSsX5wB8v0CKSbjS-WxV2zg@mail.gmail.com>
Subject: Re: [DISCUSS] FLIP-23 Model Serving
To: "dev@flink.apache.org" <dev@flink.apache.org>
Cc: Boris Lublinsky <boris.lublinsky@lightbend.com>, Eron Wright <ewright@live.com>,
	Roberto Bentivoglio <roberto.bentivoglio@radicalbit.io>,
	Riccardo Diomedi <riccardo.diomedi@radicalbit.io>,
	Mauro Cortellazzi <mauro.cortellazzi@radicalbit.io>,
	"Geerdink, A.S. (Bas)" <Bas.Geerdink@ing.nl>, Andrea Spina <andrea.spina@radicalbit.io>
Content-Type: multipart/alternative; boundary="001a1143f24a80abbb055ef7a842"
archived-at: Mon, 27 Nov 2017 14:24:33 -0000

--001a1143f24a80abbb055ef7a842
Content-Type: text/plain; charset="UTF-8"

Hi Stavros,

thanks for the detailed FLIP!
Model serving is an important use case and it's great to see efforts to add
a library for this to Flink!

I've read the FLIP and would like to ask a few questions and make some
suggestions.

1) Is it a strict requirement that a ML pipeline must be able to handle
different input types?
I understand that it makes sense to have different models for different
instances of the same type, i.e., same data type but different keys. Hence,
the key-based joins make sense to me. However, couldn't completely
different types be handled by different ML pipelines or would there be
major drawbacks?

2) I think from an API point of view it would be better to not require
input records to be encoded as ProtoBuf messages. Instead, the model server
could accept strongly-typed objects (Java/Scala) and (if necessary) convert
them to ProtoBuf messages internally. In case we need to support different
types of records (see my first point), we can introduce a Union type (i.e.,
an n-ary Either type). I see that we need some kind of binary encoding
format for the models but maybe also this can be designed to be pluggable
such that later other encodings can be added.

3) I think the DataStream Java API should be supported as a first class
citizens for this library.

4) For the integration with the DataStream API, we could provide an API
that receives (typed) DataStream objects, internally constructs the
DataStream operators, and returns one (or more) result DataStreams. The
benefit is that we don't need to change the DataStream API directly, but
put a library on top. The other libraries (CEP, Table, Gelly) follow this
approach.

5) I'm skeptical about using queryable state to expose metrics. Did you
consider using Flink's metrics system [1]? It is easily configurable and we
provided several reporters that export the metrics.

What do you think?
Best, Fabian

[1]
https://ci.apache.org/projects/flink/flink-docs-release-1.3/monitoring/metrics.html

2017-11-23 12:32 GMT+01:00 Stavros Kontopoulos <st.kontopoulos@gmail.com>:

> Hi guys,
>
> Let's discuss the new FLIP proposal for model serving over Flink. The idea
> is to combine previous efforts there and provide a library on top of Flink
> for serving models.
>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-23+-+Model+Serving
>
> Code from previous efforts can be found here: https://github.com/FlinkML
>
> Best,
> Stavros
>

--001a1143f24a80abbb055ef7a842--