flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Fabian Hueske <fhue...@gmail.com>
Subject Re: Potential bug in Flink SQL HOP and TUMBLE operators
Date Mon, 17 Sep 2018 15:54:03 GMT
Hi John,

Are you sure that the first rows of the first window are dropped?
When a query with processing time windows is terminated, the last window is
not computed. This in fact intentional and does not apply to event-time
windows.

Best, Fabian


2018-09-17 17:21 GMT+02:00 John Stone <elliotstone@gmail.com>:

> Hello,
>
> I'm checking if this is intentional or a bug in Apache Flink SQL (Flink
> 1.6.0).
>
> I am using processing time with a RocksDB backend.  I have not checked if
> this issue is also occurring in the Table API.  I have not checked if this
> issue also exists for event time (although I suspect it does).
>
> Consider the following two queries:
>
> "SELECT foo, COUNT(bar)
> FROM MyTable
> WHERE  faz = 'xyz'
> GROUP BY HOP(myTime, INTERVAL '1' SECOND, INTERVAL '5' SECOND), foo"
>
> and
>
> "SELECT foo, COUNT(bar)
> FROM MyTable
> WHERE  faz = 'xyz'
> GROUP BY TUMBLE(myTime, INTERVAL '5' SECOND), foo"
>
> I have found in my testing for both of the above that events received in
> the first 5 seconds are ignored.  In other words, the first window interval
> is effectively a black hole, and only events which are received starting
> after the first 5 seconds of the stream being "up" are processed.
>
> Is this ignoring of events during the first interval a bug or intentional?
>
> Many thanks,
>
> John
>

Mime
View raw message