Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id E0E62200D42 for ; Fri, 17 Nov 2017 19:36:56 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id DF60F160BFB; Fri, 17 Nov 2017 18:36:56 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id AFC0C160BE6 for ; Fri, 17 Nov 2017 19:36:55 +0100 (CET) Received: (qmail 40998 invoked by uid 500); 17 Nov 2017 18:36:54 -0000 Mailing-List: contact user-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list user@flink.apache.org Received: (qmail 40988 invoked by uid 99); 17 Nov 2017 18:36:54 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 17 Nov 2017 18:36:54 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id BC213C697E for ; Fri, 17 Nov 2017 18:36:53 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 3.494 X-Spam-Level: *** X-Spam-Status: No, score=3.494 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FORGED_MUA_MOZILLA=1.596, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=yahoo.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id r_WOnRdjVdk9 for ; Fri, 17 Nov 2017 18:36:52 +0000 (UTC) Received: from sonic303-23.consmr.mail.gq1.yahoo.com (sonic303-23.consmr.mail.gq1.yahoo.com [98.137.64.204]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id 219255F298 for ; Fri, 17 Nov 2017 18:36:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1510943804; bh=FAsENMTshlLVlIQcAlCNkL2UUmbtoVfhaktNVRPGG8M=; h=Date:From:Reply-To:To:Cc:In-Reply-To:References:Subject:From:Subject; b=VNKZkos57A3DqwcJGTATAHBHb8lc4thIgTBQhqyKAw2DUwQfHRNRN0yOSb79NqFfRpCE3UsiB+vWICbeiYVoevAnMcoVCDvMQkL3Yl9NwUKu2V92hRujpPZvvmkANmhQbxuUYnCZN6bXWldkDPeR7nsMWJ6ebg3+qlAl1yU0obp//x1RHfs/3ooAwFZ+kn52rM5OrtIlJDy+LMZMfmgB++ageVo8awYBjSdSlL8dN5BBpXvoAz0tUFxJTZnnuc9gP6ruaV32N+r2xhIty8Wj9cvgFqel8NIr6FQDwOKcHl1E7wExoIeTUateuv0h0gd7lA/Dm/wMkzpnRRv36tuqsw== X-YMail-OSG: eCftH.QVM1kCNTzXNTi63uFkBSMEubCGWTc6OZ_s07fukP1vHLelPWdO_mZ65eH 8EGBwiUa7EbF4rv5cuVsHXcn5J9xXoDBkNvCW7KP86Lpg13rjBuI0POddw0GK0RxHMPxCmGGjQZ9 dArtvguEsgLRrbtrTIgQD2JejQWXmDP2cIXgrcho56irlTDTpa0MpR7v.Lsd6pnzd3p1mtr8b1bI D_sohmPGZnhcrhgTzkLFp9McPrA.yg5uT0gyl_kUvu16Qk4QEZef43mKFRDVzaixEZQ5lzIBiyxD Z4tr37NRg4tgNHI1dbxSnNRwl2gldteb3UrHt7kcMVz2210xKJqWXJHBCdfFDagVmk9_eI11jCy8 j.U.npYfprA9cMvcGq6.8yCU8F2EVR4_Ft9fJoVcm8Ub7cidY4sPZ.bLqEcpWFi5uaj681eMtUCG Yj0oTDZM1HQlrzawSjTusRPA8teSkHUlKS4OOMZayS0.pHZrYlthv1Ch1qsBl2XzoPGTWHLGZwpo p2cKH7aiBPg-- Received: from sonic.gate.mail.ne1.yahoo.com by sonic303.consmr.mail.gq1.yahoo.com with HTTP; Fri, 17 Nov 2017 18:36:44 +0000 Date: Fri, 17 Nov 2017 18:36:43 +0000 (UTC) From: M Singh Reply-To: M Singh To: M Singh , Aljoscha Krettek , Stefan Richter Cc: User Message-ID: <1581585301.393698.1510943803028@mail.yahoo.com> In-Reply-To: <1586898164.394985.1510943582630@mail.yahoo.com> References: <141547276.1216499.1510600760432.ref@mail.yahoo.com> <141547276.1216499.1510600760432@mail.yahoo.com> <97E6E5A9-23FC-4F66-8331-DA5D3672A463@data-artisans.com> <1586898164.394985.1510943582630@mail.yahoo.com> Subject: Re: Apache Flink - Question about TriggerResult.FIRE MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_393697_929613375.1510943803025" X-Mailer: WebService/1.1.10976 YahooMailNeo Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36 archived-at: Fri, 17 Nov 2017 18:36:57 -0000 ------=_Part_393697_929613375.1510943803025 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Also, Stefan - You mentioned=C2=A0 "In the first case, it is a new window without the previous elements, in th= e second case the window reflects the old contents plus all changes since t= he last trigger." I am assuming the first case is FIRE and second case is FIRE_AND_PURGE - I = was thinking that in the first case (FIRE), there would be elements from pr= evious window since we did not purge and the second case it would have only= new elements since it was purged.=C2=A0 Am I missing something ?=C2=A0=C2= =A0 Once again, if there are any examples it would greatly help me in understan= ding the semantics/usage scenarios. Mans=20 On Friday, November 17, 2017 10:33 AM, M Singh w= rote: =20 Thanks Stefan and Aljoscha for your responses. Stefan - When I mentioned "new window" - I meant the next window being crea= ted.=C2=A0 Eg:=C2=A0 if the event was in w1 based processing time and the trigger retu= rned FIRE - then after the window function is computed, what happens to the= events in that window (w1).=C2=A0 Are they (elements in w1) propagated to = the next processing time window (w2) ?=C2=A0 If not, then what is the diffe= rence between FIRE and FIRE_AND_PURGE and when do we use FIRE vs FIRE_AND P= URGE ?=C2=A0 Are there any examples to demonstrated these differences ? Thanks again for your help.=C2=A0 Mans=20 On Thursday, November 16, 2017 5:16 AM, Aljoscha Krettek wrote: =20 Yes, all of this is correct. Sliding windows in fact look like completely = separate windows to the windowing system. Best,Aljoscha On 16. Nov 2017, at 10:15, Stefan Richter wro= te: Hi, I think the effect is pretty straight forward, the elements in a window are= not purged if the trigger is only FIRE and not FIRE_AND_PURGE. Unfortunate= ly, your question is a bit unclear about what exactly you mean by =E2=80=9E= new window=E2=80=9C: a truly =E2=80=9Enew=E2=80=9C window or another trigge= ring of the previous (non-purged) window? In the first case, it is a new wi= ndow without the previous elements, in the second case the window reflects = the old contents plus all changes since the last trigger. For sliding windows, if I remember correctly, every slide is actually a dif= ferent window and elements are just added repeatedly to all windows in whic= h they belong. So window n+1 should not be affected by whether or not windo= w n purges or not. Maybe Aljoscha (in CC) can confirm this for us. Best,Stefan Am 13.11.2017 um 20:19 schrieb M Singh : Hi Flink Users I have a few questions about triggers: If a trigger returns TriggerResult.FIRE from say the onProcessingTime metho= d - the window computation is triggered but elements are kept in the window= .=C2=A0 If there a second invocation of the onProcessingTime method will th= e elements from the previous window (which were not purged) a part of the n= ew window computation along with new events added since the last FIRE event= ?=C2=A0 Secondly, how does the FIRE option affect the sliding window computation ? If there are any other insights/pitfalls while dealing with this, please le= t me know. Thanks Mans =20 =20 ------=_Part_393697_929613375.1510943803025 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
<= span id=3D"yui_3_16_0_ym19_1_1510943263131_16320">Also, Stefan - You mentio= ned 

<= /span>
<= span style=3D"font-family: "Helvetica Neue", "Segoe UI"= , Helvetica, Arial, "Lucida Grande", sans-serif;" id=3D"yui_3_16_= 0_ym19_1_1510943263131_16324">"In the first case, it is a new window withou= t the previous elements, in the second case the window reflects the old con= tents plus all changes since the last trigger."

I am as= suming the first case is FIRE and second case is FIRE_AND_PURGE - I was thi= nking that in the first case (FIRE), there would be elements from previous = window since we did not purge and the second case it would have only new el= ements since it was purged.  Am I missing something ?  

Once again, if there are any = examples it would greatly help me in understanding the semantics/usage scen= arios.

Mans


On Friday, November 17, 2017 10:33 AM, M Singh &l= t;mans2singh@yahoo.com> wrote:


Thanks Stefan and Aljoscha for your responses.

Stefan - When I mentioned "new window" - I meant the next window being= created. 

Eg:  if the event was in w1 based proc= essing time and the trigger returned FIRE - then after the window function = is computed, what happens to the events in that window (w1).  Are they= (elements in w1) propagated to the next processing time window (w2) ? = ; If not, then what is the difference between FIRE and FIRE_AND_PURGE and w= hen do we use FIRE vs FIRE_AND PURGE ?  Are there any examples to demo= nstrated these differences ?

Thanks again for your h= elp. 

Mans


On Thursday, November 16, 2017 5:16 AM, Aljoscha Krettek &= lt;aljoscha@apache.org> wrote:


=
Yes, all of this is correct. Slidi= ng windows in fact look like completely separate windows to the windowing s= ystem.

Best,
Aljoscha=

On 16. Nov= 2017, at 10:15, Stefan Richter <s.richter@data-arti= sans.com> wrote:

= Hi,

=
I think the effect is pretty straight fo= rward, the elements in a window are not purged if the trigger is only FIRE = and not FIRE_AND_PURGE. Unfortunately, your question is a bit unclear about= what exactly you mean by =E2=80=9Enew window=E2=80=9C: a truly =E2=80=9Ene= w=E2=80=9C window or another triggering of the previous (non-purged) window= ? In the first case, it is a new window without the previous elements, in t= he second case the window reflects the old contents plus all changes since = the last trigger.

For sliding windows,= if I remember correctly, every slide is actually a different window and el= ements are just added repeatedly to all windows in which they belong. So wi= ndow n+1 should not be affected by whether or not window n purges or not. M= aybe Aljoscha (in CC) can confirm this for us.

Best,
Stefan

Am 13.11.2017 um 20:19 schrieb M Singh <mans2singh@y= ahoo.com>:

<= div class=3D"yiv3574198830" style=3D"background-color:rgb(255, 255, 255);fo= nt-family:'Helvetica Neue', Helvetica, Arial, 'Lucida Grande', sans-serif;f= ont-size:13px;">
Hi Flink Users

I have a few questions about tri= ggers:

If a trigger returns TriggerResult.FIRE from say the on= ProcessingTime method - the window computation is triggered but elements ar= e kept in the window.  If there a second invocation of the onProcessin= gTime method will the elements from the previous window (which were not pur= ged) a part of the new window computation along with new events added since= the last FIRE event ? 

Secondly, how does t= he FIRE option affect the sliding window computation ?

If there are any other i= nsights/pitfalls while dealing with this, please let me know.

Thanks

Mans






=


= ------=_Part_393697_929613375.1510943803025--