flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vishnu Viswanath <vishnu.viswanat...@gmail.com>
Subject Re: Firing windows multiple times
Date Wed, 10 Aug 2016 17:26:06 GMT
Hi Aljoscha,

This looks like the bug that we discussed, as part of Enhance window
evictor JIRA


On Wed, Aug 10, 2016 at 1:18 PM, Aljoscha Krettek <aljoscha@apache.org>

> Hi,
> from your mail I'm gathering that you are in fact using an Evictor, is
> that correct? If not, then the window operator should not keep all the
> elements ever received for a window but only the aggregated result.
> Side note, there seems to be a bug in EvictingWindowOperator that causes
> evicted elements to not actually be removed from the state. They are only
> filtered from the Iterable that is given to the WindowFunction. I opened a
> Jira issue for that: https://issues.apache.org/jira/browse/FLINK-4369
> Cheers,
> Aljoscha
> On Wed, 10 Aug 2016 at 18:19 Shannon Carey <scarey@expedia.com> wrote:
>> One unfortunate aspect of using a fold() instead of a window is that the
>> fold function has no knowledge of the watermarks. As a result, it is
>> difficult to ensure that only items before the current watermark are
>> included in the aggregation, and that old items are evicted correctly. This
>> fact lends more support to the idea of using a custom operator (though that
>> is more complex) or adding support for this use case within Flink.
>> -Shannon

View raw message