flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Simone Robutti <simone.robu...@radicalbit.io>
Subject Re: Using FlinkML algorithms in Streaming
Date Wed, 11 May 2016 14:23:49 GMT
Actually model portability and persistence is a serious limitation to
practical use of FlinkML in streaming. If you know what you're doing, you
can write a blunt serializer for your model, write it in a file and rebuild
the model stream-side with deserialized informations.

I tried it for an SVM model and there were no obstacles. It's ugly but it
works.

2016-05-11 16:18 GMT+02:00 Márton Balassi <balassi.marton@gmail.com>:

> Currently I am not aware of streaming learners support, you would need to
> implement that yourself at this point.
>
> As for streaming predictors for batch learners I have some preview code
> that you might like. [1]
>
> [1]
> https://github.com/streamline-eu/ML-Pipelines/blob/314e3d940f1f1ac7b762ba96067e13d806476f57/flink-libraries/flink-stream-ml/src/main/scala/org/apache/flink/stream/ml/examples/MLRExample.scala
>
>
>
> On Wed, May 11, 2016 at 3:52 PM, Piyush Shrivastava <piyushjoy@yahoo.co.in
> > wrote:
>
>> Hi Márton,
>>
>> I want to train and get the residuals.
>>
>> Thanks and Regards,
>> Piyush Shrivastava <piyush@webograffiti.com>
>> [image: WeboGraffiti]
>> http://webograffiti.com
>>
>>
>> On Wednesday, 11 May 2016 7:19 PM, Márton Balassi <
>> balassi.marton@gmail.com> wrote:
>>
>>
>> Hey Piyush,
>>
>> Would you like to train or predict on the streaming data?
>>
>> Best,
>>
>> Marton
>>
>> On Wed, May 11, 2016 at 3:44 PM, Piyush Shrivastava <
>> piyushjoy@yahoo.co.in> wrote:
>>
>> Hello all,
>>
>> I want to perform linear regression using FlinkML's
>> MultipleLinearRegression() function on streaming data.
>>
>> This function takes a DataSet as an input and I cannot create a DataSet
>> inside the MapFunction of a DataStream. How can I use this function on my
>> DataStream?
>>
>> Thanks and Regards,
>> Piyush Shrivastava <piyush@webograffiti.com>
>> [image: WeboGraffiti]
>> http://webograffiti.com
>>
>>
>>
>>
>>
>

Mime
View raw message