flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Márton Balassi <balassi.mar...@gmail.com>
Subject Re: FlinkML and DataStream API
Date Wed, 21 Dec 2016 21:44:05 GMT
Thanks for mentioning it, Theo.

Here it is: https://github.com/streamline-eu/ML-Pipelines/tree/stream-ml

Look at these examples:
https://github.com/streamline-eu/ML-Pipelines/commit/314e3d940f1f1ac7b762ba96067e13d806476f57

On Wed, Dec 21, 2016 at 9:38 PM, <dromitlabs@gmail.com> wrote:

> I'm interested in that code you mentioned too, I hope you can find it.
>
> Regards,
> Matt
>
> On Dec 21, 2016, at 17:12, Theodore Vasiloudis <
> theodoros.vasiloudis@gmail.com> wrote:
>
> Hello Mäki,
>
> I think what you would like to do is train a model using batch, and use
> the Flink streaming API as a way to serve your model and make predictions.
>
> While we don't have an integrated way to do that in FlinkML currently, I
> definitely think that's possible. I know Marton Balassi has been working on
> something like this for the ALS algorithm, but I can't find the code right
> now on mobile.
> The general idea is to keep your model as state and use it to make
> predictions on a stream of incoming data.
>
> Model serving is definitely something we'll be working on in the future,
> I'll have a master student working on exactly that next semester.
>
> --
> Sent from a mobile device. May contain autocorrect errors.
>
> On Dec 21, 2016 5:24 PM, "Mäki Hanna" <Hanna.Maki@comptel.com> wrote:
>
>> Hi,
>>
>>
>>
>> I’m wondering if there is a way to use FlinkML and make predictions
>> continuously for test data coming from a DataStream.
>>
>>
>>
>> I know FlinkML only supports the DataSet API (batch) at the moment, but
>> is there a way to convert a DataStream into DataSets? I’m thinking of
>> something like
>>
>>
>>
>> (0. fit model in batch mode)
>>
>> 1. window the DataStream
>>
>> 2. convert the windowed stream to DataSets
>>
>> 3. use the FlinkML methods to make predictions
>>
>>
>>
>> BR,
>>
>> Hanna
>>
>>
>> Disclaimer: This message and any attachments thereto are intended solely
>> for the addressed recipient(s) and may contain confidential information. If
>> you are not the intended recipient, please notify the sender by reply
>> e-mail and delete the e-mail (including any attachments thereto) without
>> producing, distributing or retaining any copies thereof. Any review,
>> dissemination or other use of, or taking of any action in reliance upon,
>> this information by persons or entities other than the intended
>> recipient(s) is prohibited. Thank you.
>>
>

Mime
View raw message