flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Biplob Biswas <revolutioni...@gmail.com>
Subject Re: Flink CEP not emitting timed out events properly
Date Mon, 26 Jun 2017 14:34:58 GMT
Hi Kostas,

I ended up setting my 
currentMaxTimestamp = Math.max(timestamp, currentMaxTimestamp); 
to
currentMaxTimestamp = Math.min(timestamp, currentMaxTimestamp); 

and  changing this :
if(firstEventFlag && (currentTime - systemTimeSinceLastModification >
10000)){ 
      systemTimeSinceLastModification = currentTime; 
      currentMaxTimestamp = currentMaxTimestamp + 1000; 
      //log.info("Current Max Time - {}, Last Modification Time - {}",
currentMaxTimestamp, systemTimeSinceLastModification ); 
    } 

to

if(firstEventFlag && (currentTime - systemTimeSinceLastModification >
20000)){ 
      systemTimeSinceLastModification = currentTime; 
      currentMaxTimestamp = currentMaxTimestamp + 10000; 
    } 

it is working fine now such that I have all the events ... with timeout and
with matches. Although I am afraid, this might not be the best way to do
things(I am still investigating what I can do and change) because this
change from max to min can lead to changes in the watermark which is not
just ascending but descending as well (that's what I think can happen when
an event with lower timestamp than the current watermark arrives) .... and
from whatever I have read so far, watermarks should always move forward....
I haven't had any such behaviour so far but if it happens what should I
expect? My job blowing up or some undefined behaviour? 

Any inputs would be helpful.

BR,
Biplob



--
View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-CEP-not-emitting-timed-out-events-properly-tp13794p13999.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.

Mime
View raw message