flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Taher Koitawala <taher.koitaw...@gslab.com>
Subject Re: Watermarks in Event Time windowing
Date Fri, 14 Sep 2018 04:08:24 GMT
Yes in many cases what we have faced that let's say in a keyed stream an
element of a specific key comes in which triggers a new window. If a
corresponding elements of the same key does not arrive a new watermark is
not generated for the window to purge. Then we faced issues with flink
keeping records in the window and not purging them at all. For this reason
we explicitly had to write some logic to generate a timestamp this is
greater than the currentTimestamp and then forcefully make the window purge
its elements.



On 14-Sep-2018 7:48 AM, "vino yang" <yanghua1127@gmail.com> wrote:

Hi Taher,

For some questions, I suggest you read the documentation related to Flink
EventTime first, for example [1]

About this question:
What happens if the watermark is same as the timestamp?

Here "timestamp", do you mean the current timestamp of Processing time? If
that's the best, it's an ideal state, and it's often difficult to achieve
this state, because eventtime always has more or less delay as events are
transmitted from the source to the processing system.

Thanks, vino.

[1]:
https://ci.apache.org/projects/flink/flink-docs-release-1.6/dev/event_timestamps_watermarks.html

Taher Koitawala <taher.koitawala@gslab.com> 于2018年9月14日周五 上午2:07写道:

> Hi All,
>           Can someone show a good example of how watermarks need to be
> generated when using EventTime windows?  What happens if the watermark is
> same as the timestamp?  How does the watermark help in the window to be
> triggered and what if watermarks are kept behind the currentTimestamps in
> the assignTimestampsAndWatermarks function.
>
>
>

Mime
View raw message