flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aljoscha Krettek <aljos...@apache.org>
Subject Re: Flink streaming connect and split streams
Date Fri, 01 Jul 2016 15:56:25 GMT
I'm afraid the only way to do it right now is using the wrapper that can
contain both, as you suggested.


On Thu, 30 Jun 2016 at 16:50 Martin Neumann <mneumann@sics.se> wrote:

> Hej,
> I'm currently playing around with some machine learning algorithms in
> Flink streaming.
> I have an input stream that I partition by key and then do a map on each
> of the keys, feeding a model and producing a prediction output.
> Periodically each operator needs to send model updates to all other
> operators.
> What is the best way to implement the structure?
> My current idea is to use the CoMap function as operator. The first stream
> is the raw data the second stream the model updates which I could just
> broadcast from the iterative stream. My problem right now is that I need
> the CoMap to basically have 2 Streams as output the model updates and the
> prediction results.
> I could write a wrapper class containing both output types but that would
> require me to separate them afterwards. This feels very clunky, is there a
> better way of dealing with this?
> cheers Martin

View raw message