flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Till Rohrmann <trohrm...@apache.org>
Subject Re: Prioritize DataStream
Date Tue, 22 Aug 2017 07:57:52 GMT
Hi Elias,

sorry for the slow answer. You were right that the answer is currently no.

However, people are currently working on changing the way the stream
operators work. This will allow the operator to decide from which input to
read next. Having such a functionality will enable us to implement proper
side inputs which will also cover your use case.

At the moment, one thing you could try is to buffer events in operator
state until you have fully consumed the other stream. Upon seeing all
events from the other stream (needs a proper termination event you can
detect) you can then replay all the buffered events. This could however
entail quite a big user state.

Cheers,
Till

On Mon, Aug 21, 2017 at 8:14 PM, Elias Levy <fearsome.lucidity@gmail.com>
wrote:

> Flink folks,
>
> A response to the question below?
>
> On Sat, Aug 19, 2017 at 11:02 AM, Elias Levy <fearsome.lucidity@gmail.com>
> wrote:
>
>> I believe the answer to this question is "no", but I figure I might as
>> well ask.  Is there a way to prioritize a stream?
>>
>> The use case is prioritization of a control stream.  This is mostly
>> needed on start-up, where a job might start consuming from the data stream
>> before consuming from the control stream.
>>
>>
>>
>

Mime
View raw message