flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Till Rohrmann <trohrm...@apache.org>
Subject Re: Watermark for each key?
Date Wed, 24 Apr 2019 17:00:21 GMT
Hi Lasse,

at the moment this is not supported out of the box by Flink. The community
thought about this feature but so far did not implement it. Unfortunately,
I'm also not aware of an easy workaround one could do in the user code
space.

Cheers,
Till

On Wed, Apr 24, 2019 at 3:26 PM Lasse Nedergaard <lassenedergaard@gmail.com>
wrote:

> Hi.
>
> We work with IoT data and we have cases where the IoT-device delay data
> transfer if it can't get network access. We would like to use table windows
> aggregate function over each device to calculate some statistics, but for
> windows aggregate functions to work we need to assign a watermark. This
> watermark is general for all devices. We can set allow latency, but we
> can't set it to months.
> So what we need is to have a watermark for each device (key by) so the
> window aggregate work on the timestamp delivered for the device and not the
> global watermark.
> Is that possible, or have anyone consider this feature?
>
> Best
>
> Lasse Nedergaard
>
>

Mime
View raw message