flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aljoscha Krettek <aljos...@apache.org>
Subject Re: Flink streaming connect and split streams
Date Fri, 01 Jul 2016 15:56:25 GMT
Hi,
I'm afraid the only way to do it right now is using the wrapper that can
contain both, as you suggested.

Cheers,
Aljoscha

On Thu, 30 Jun 2016 at 16:50 Martin Neumann <mneumann@sics.se> wrote:

> Hej,
>
> I'm currently playing around with some machine learning algorithms in
> Flink streaming.
>
> I have an input stream that I partition by key and then do a map on each
> of the keys, feeding a model and producing a prediction output.
> Periodically each operator needs to send model updates to all other
> operators.
>
> What is the best way to implement the structure?
>
> My current idea is to use the CoMap function as operator. The first stream
> is the raw data the second stream the model updates which I could just
> broadcast from the iterative stream. My problem right now is that I need
> the CoMap to basically have 2 Streams as output the model updates and the
> prediction results.
>
> I could write a wrapper class containing both output types but that would
> require me to separate them afterwards. This feels very clunky, is there a
> better way of dealing with this?
>
> cheers Martin
>

Mime
View raw message