flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dominik Bruhn <domi...@dbruhn.de>
Subject TimeWindow Trigger which only fires when the values have changed
Date Tue, 04 Oct 2016 12:54:30 GMT
Hi,
I'm heavily relying on TimeWindows for my real time processing. Roughly 
my job consumes from an AMQP queue, computes some time buckets and saves 
the time-buckets to Cassandra.

I found the EventTimeTriggerWithEarlyAndLateFiring [1] class which 
already helped me a lot: Even with long time-windows, I can get 
intermediate values already saved to Cassandra by using the earlyFiring 
(and setting "accumulating" to true.

My question is: Would it be possible to only write fire the trigger if 
the value of the TimeBucket has changed? What I actually want is only 
writing to Cassandra if there is actually something different in the 
time bucket.

And, as a side question: Why is something like the 
EventTimeTriggerWithEarlyAndLateFiring not in the default Flink 
distribution? It seems very handy.

[1]: 
https://github.com/dataArtisans/beam_comp/blob/master/src/main/java/com/dataartisans/beam_comparison/customTriggers/EventTimeTriggerWithEarlyAndLateFiring.java


Thanks,
Dominik

Mime
View raw message