flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "JingsongLee" <lzljs3620...@aliyun.com>
Subject 回复:Is it possible to handle late data when using table API?
Date Tue, 16 Apr 2019 09:08:52 GMT
Hi @Lasse Nedergaard, Table API don't have allowedLateness api.
But you can set rowtime.watermarks.delay of source to slow down the watermark clock.

发件人:Lasse Nedergaard <lassenedergaard@gmail.com>
发送时间:2019年4月16日(星期二) 16:20
收件人:user <user@flink.apache.org>
主 题:Is it possible to handle late data when using table API?


I have a simple tumble window working on eventtime like this.

Table res30MinWindows = machineInsights
        .window(Tumble.over("30.minutes").on("UserActionTime").as("w")) // define window
        .groupBy("machineId, machineInsightId, w") // group by key and window
        .select("machineId, machineInsightId, w.start, w.end, w.rowtime, value.max as max");
// access window properties and aggregate
As we work with Iot units we don't have 100% control over the eventtime reported and therefore
need to handle late data to ensure that we don't do our calculation wrong.
I would like to know if there is any option in the Table API to get access to late data, or
my only option is to use Streaming API?
Thanks in advance
Lasse Nedergaard

View raw message