flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Elias Levy <fearsome.lucid...@gmail.com>
Subject TimeWindow overload?
Date Tue, 03 May 2016 02:07:38 GMT
Looking over the code, I see that Flink creates a TimeWindow object each
time the WindowAssigner is created.  I have not yet tested this, but I am
wondering if this can become problematic if you have a very long sliding
window with a small slide, such as a 24 hour window with a 1 minute slide.
It seems this would create 1,440 TimeWindow objects per event.  Event a low
event rates this would seem to result in an explosion of TimeWindow
objects: at 1,000 events per second, you'd be creating 1,440,000 TImeWindow
objects.  After 24 hours you'd have nearly 125 billion TM objects that
would just begin to be purged.

Does this analysis seem right?

I suppose that means you should not use long length sliding window with
small slides.

View raw message