flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jamie Grier <ja...@data-artisans.com>
Subject Re: Hi, There is possibly an issue with EventTimeSessionWindows where a gap is specified for considering items in the same session. Here the logic is, if two adjacent items have a difference in event timestamps of more than the gap then the items are considered to be in separate session. The issue is, what happens if the gap between streaming records is never (or for a very long time) less than the session gap. This is likely to lead to a race condition. Is this a bug? How do we deal with this to process windows in finite time? Please could you suggest. Thanks.
Date Mon, 02 Jan 2017 19:54:21 GMT
If there is never a gap between elements larger than the session gap -- the
window never ending would be the correct behavior.  So, if this is the case
with some data stream I would not suggest to use session windows at all --
or I would use a smaller session gap.

Another alternative would be to use Session Windows along with a
user-defined trigger that fires periodically whether the session has ended
or not.  For example, if in the normal case the session window logic works
well but sometimes you want to force an eval in case a "natural" session is
too long.

-Jamie

On Mon, Jan 2, 2017 at 3:11 AM, Sujit Sakre <sujit.sakre@northgateps.com>
wrote:

> Hi,
>
> We are using Flink 1.1.4 version.
>
>
> There is possibly an issue with EventTimeSessionWindows where a gap is
> specified for considering items in the same session. Here the logic is, if
> two adjacent items have a difference in event timestamps of more than the
> gap then the items are considered to be in separate session. The issue is,
> what happens if the gap between streaming records is *never*  (or for a
> very long time) less than the session gap. This is likely to lead to a race
> condition.
>
> Is this a bug? How do we deal with this to process windows in finite time?
>
> Please could you suggest.
>
> Thanks.
>
>
> *Sujit Sakre*
>
> Senior Technical Architect
> Tel: +91 22 6660 6600
> Ext:
> 247
> Direct: 6740 5247
>
> Mobile: +91 98672 01204
>
> www.rave-tech.com
>
>
>
> Follow us on: Twitter <https://twitter.com/Rave_Tech> / LinkedIn
> <https://in.linkedin.com/in/ravetechnologies> / YouTube
> <https://www.youtube.com/channel/UCTaO1am-cm4FqnQCGdB6ExA>
>
>
>
> Rave Technologies – A Northgate Public Services Company
> <https://www.google.co.in/maps/place/Rave+Technologies/@19.0058078,72.823516,17z/data=!3m1!4b1!4m5!3m4!1s0x3bae17fcde71c3b9:0x1e2a8c0c4a075145!8m2!3d19.0058078!4d72.8257047>
>
>
>
> Please consider the environment before printing this email
>
> This email is sent on behalf of Northgate Public Services (UK) Limited and
> its associated companies including Rave Technologies (India) Pvt Limited
> (together "Northgate Public Services") and is strictly confidential and
> intended solely for the addressee(s).
> If you are not the intended recipient of this email you must: (i) not
> disclose, copy or distribute its contents to any other person nor use its
> contents in any way or you may be acting unlawfully;  (ii) contact
> Northgate Public Services immediately on +44(0)1908 264500
> <+44%201908%20264500> quoting the name of the sender and the addressee
> then delete it from your system.
> Northgate Public Services has taken reasonable precautions to ensure that
> no viruses are contained in this email, but does not accept any
> responsibility once this email has been transmitted.  You should scan
> attachments (if any) for viruses.
>
> Northgate Public Services (UK) Limited, registered in England and Wales
> under number 00968498 with a registered address of Peoplebuilding 2,
> Peoplebuilding Estate, Maylands Avenue, Hemel Hempstead, Hertfordshire, HP2
> 4NN.  Rave Technologies (India) Pvt Limited, registered in India under
> number 117068 with a registered address of 2nd Floor, Ballard House, Adi
> Marzban Marg, Ballard Estate, Mumbai, Maharashtra, India, 400001.
>



-- 

Jamie Grier
data Artisans, Director of Applications Engineering
@jamiegrier <https://twitter.com/jamiegrier>
jamie@data-artisans.com

Mime
View raw message