flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dominik Bruhn <domi...@dbruhn.de>
Subject BoundedOutOfOrdernessTimestampExtractor and timestamps in the future
Date Tue, 01 Nov 2016 14:05:25 GMT
I'm using a BoundedOutOfOrdernessTimestampExtractor for assigning my 
timestamps and discarding to old events (which happens sometimes).

Now my problem is that some events, by accident have timestamps in the 
future. If the timestamps are more in the future than my 
`maxOutOfOrderness`, I'm discarding valid events. So I need a way of 
saying that the
BoundedOutOfOrdernessTimestampExtractor should exclude timestamps from 
the future for the watermark calculation. I still want to keep the 
events if they are in the future and assign them to the right 

How can I achieve this? I thought about checking whether the potential 
timestamp is in the future before considering it for a watermark. I 
cloned the BoundedOutOfOrdernessTimestampExtractor and added the idea

Does this make sense? Or is there a better approach?

In general, how does Flink handle readings from the future?



View raw message