flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Maximilian Michels <...@apache.org>
Subject Re: Apache Flink: aligning watermark among parallel tasks
Date Tue, 29 Mar 2016 11:58:09 GMT
Hi Ozan,

You probably want to look at a custom Trigger implementation. Please see
the different triggers in
org/apache/flink/streaming/api/windowing/triggers/. You can write your own
event-based trigger. Best thing would be to extend the EventTimeTrigger
with your logic.

Then you can use windowedStream.trigger(new MyCustomTrigger()) to include
your trigger logic on your windowed stream.

Cheers,
Max


On Thu, Mar 24, 2016 at 10:51 AM, Ozan DENÄ°Z <ozandeniz@outlook.com> wrote:

> We are using periodic event time window with watermark. We have currently
> 4 parallel tasks in our Flink App.
>
> During the streaming process, all the 4 tasks' watermark values must be
> close to trigger window event.
>
>
>
>
>
>   For example;
>
>
>   Task 1 watermark value = 8
>
>
>   Task 2 watermark value = 1
>
>
>   Task 3 watermark value = 8
>
>
>   Task 4 watermark value = 8
>
>
>
>
>
> Task 2 is waiting for log to update its watermark. However,
> the condition can occur before Task 2's update and we want to fire the
> window event before it.
>
>
>
>
> Is there any mechanism to align all the parallel tasks' watermarks or fire
> the window event without waiting for other tasks?
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message