flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From rhashmi <rizhas...@hotmail.com>
Subject Re: State in Custom Tumble Window Class
Date Tue, 30 May 2017 11:43:18 GMT
Thanks Aljoscha Krettek, 

So the results will not be deterministic for late events. For idempotent
update, i would need to find an additional key base of current event time if
they are late and attached to the aggregator which probably possible by
doing some function(maxEventTime, actualEventTime). For that i need
maxEventTime to be stored as part of state & recover in case of runtime
failure.

Here is my corner case like. 
-- If assume whole flink runtime crashed(auto commit on) & after recovery
the first event arrived is from past(actually late). Without keeping max
currentTime state, may potentially override previous aggregate. 

I was wondering if i can record my last max EventTime as part of checkPoint, 
or run query against sink source to find last processed eventtime.

Any recommendation?
 



--
View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/State-in-Custom-Tumble-Window-Class-tp13177p13387.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.

Mime
View raw message