Mailing-List: contact user-help@flink.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@flink.apache.org
MIME-Version: 1.0
In-Reply-To: <C79A5E03-7F94-4051-BA9D-FA6850B96B7C@data-artisans.com>
References: <CAGq0LH2t=kKjc5SVpSTFir1c=W71t2nkw7ZHfD8XWwHup8J0eg@mail.gmail.com>
 <C069C830-2025-4B0A-BCDD-4A74B3A4ECDC@data-artisans.com> <CAGq0LH1_oV+vBjwhhhv4taeEQhmL2kTzMTttcCxMf6WsLsvNXQ@mail.gmail.com>
 <C79A5E03-7F94-4051-BA9D-FA6850B96B7C@data-artisans.com>
From: Janardhan Reddy <janardhan.reddy@olacabs.com>
Date: Wed, 30 Nov 2016 23:38:06 +0530
Message-ID: <CAGq0LH3uhfXvfOiHWo0wKzuoypinn=PPE6gXjQUMAkPG1VYKgg@mail.gmail.com>
Subject: Re: Query regarding tumbling event time windows with ingestion time
To: user@flink.apache.org
Content-Type: multipart/alternative; boundary=94eb2c0859689b19a00542889883
archived-at: Wed, 30 Nov 2016 18:08:59 -0000

--94eb2c0859689b19a00542889883
Content-Type: text/plain; charset=UTF-8

That makes it clear.

Thanks

On Wed, Nov 30, 2016 at 10:15 PM, Kostas Kloudas <
k.kloudas@data-artisans.com> wrote:

> Hello,
>
> If you have events:
>
> A@10m, B@14m, Watermark@15m
> C@16m D@25m, Watermark@30m
>
> then the result will be:
>
> [A,B]@14.59
> [C,D]@29.59
>
> This means that for the next windowing, you will have the elements
> [A,B]@14.59 and [C,D]@29.59
> to window in windows of 5min.
>
> Given that they are 15min apart, this means that you will have:
>
> [A,B]@14.59 in one window and [C,D]@29.59
>
> This is what I meant by at most one element per window, because window 0
> to 5 min, 5 to 10 and
> 15 to 20 and 20 to 25 will be empty.
>
> Does this make it clearer?
>
> Kostas
>
> On Nov 30, 2016, at 11:48 AM, Janardhan Reddy <janardhan.reddy@olacabs.com>
> wrote:
>
> HI
> i didn't get it , can you please clarify with an example in case each of
> operation A and B emit multiple elements.
>
> On Wed, Nov 30, 2016 at 3:34 PM, Kostas Kloudas <
> k.kloudas@data-artisans.com> wrote:
>
>> Hi Janardhan,
>>
>> After the first windowing operation, the timestamp of the emitted element
>> for each window
>> will be the (endOfWindow - 1). So in your case, in the second windowing
>> operation (window by 5)
>> there will be at most one element per window.
>>
>> I hope this answers your question.
>>
>> Kostas
>>
>> > On Nov 29, 2016, at 7:25 PM, Janardhan Reddy <
>> janardhan.reddy@olacabs.com> wrote:
>> >
>> > Hi,
>> >
>> > Suppose we have a stream like this.
>> >
>> > someStream.timeWindow(Time.minutes(15)).apply {
>> > operation A
>> >
>> > }.keyby("....").window(TumblingEventTimeWindows.ofseconds(5)).apply {
>> >
>> > operation B
>> >
>> > }.keyby("....").window(TumblingEventTimeWindows.ofseconds(5)).apply {
>> >
>> > operation C
>> >
>> > }
>> >
>> > Say operation A emits some elements => it would be emitted every 15
>> minutes.
>> >
>> > How would be the window behaviour of where operation B takes place if
>> operation A takes more than 5 seconds with ingestion Time characteristic.
>> Similarly how would windows behave near operation C if operation B takes
>> more than 5 seconds.
>> >
>> >
>> > Thanks
>> > Janardhan
>>
>>
>
>

--94eb2c0859689b19a00542889883
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">That makes it clear.<div><br></div><div>Thanks</div></div>=
<div class=3D"gmail_extra"><br><div class=3D"gmail_quote">On Wed, Nov 30, 2=
016 at 10:15 PM, Kostas Kloudas <span dir=3D"ltr">&lt;<a href=3D"mailto:k.k=
loudas@data-artisans.com" target=3D"_blank">k.kloudas@data-artisans.com</a>=
&gt;</span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0=
 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style=3D"word-wra=
p:break-word">Hello,<div><br></div><div>If you have events:<div><br></div><=
div>A@10m, B@14m, Watermark@15m=C2=A0</div><div>C@16m D@25m, Watermark@30m<=
/div><div><br></div><div>then the result will be:</div><div><br></div><div>=
[A,B]@14.59</div><div>[C,D]@29.59</div><div><br></div><div>This means that =
for the next windowing, you will have the elements [A,B]@14.59 and [C,D]@29=
.59</div><div>to window in windows of 5min.=C2=A0</div><div><br></div><div>=
Given that they are 15min apart, this means that you will have:</div><div><=
br></div><div>[A,B]@14.59 in one window and [C,D]@29.59</div><div><br></div=
><div>This is what I meant by at most one element per window, because windo=
w 0 to 5 min, 5 to 10 and=C2=A0</div><div>15 to 20 and 20 to 25 will be emp=
ty.</div><div><br></div><div>Does this make it clearer?</div><span class=3D=
"HOEnZb"><font color=3D"#888888"><div><br></div><div>Kostas</div></font></s=
pan><div><div class=3D"h5"><div><br></div><div><div><blockquote type=3D"cit=
e"><div>On Nov 30, 2016, at 11:48 AM, Janardhan Reddy &lt;<a href=3D"mailto=
:janardhan.reddy@olacabs.com" target=3D"_blank">janardhan.reddy@olacabs.com=
</a>&gt; wrote:</div><br class=3D"m_4387370997278467890Apple-interchange-ne=
wline"><div><div dir=3D"ltr">HI<div>i didn&#39;t get it , can you please cl=
arify with an example in case each of operation A and B emit multiple eleme=
nts.</div></div><div class=3D"gmail_extra"><br><div class=3D"gmail_quote">O=
n Wed, Nov 30, 2016 at 3:34 PM, Kostas Kloudas <span dir=3D"ltr">&lt;<a hre=
f=3D"mailto:k.kloudas@data-artisans.com" target=3D"_blank">k.kloudas@data-a=
rtisans.com</a>&gt;</span> wrote:<br><blockquote class=3D"gmail_quote" styl=
e=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi Jana=
rdhan,<br>
<br>
After the first windowing operation, the timestamp of the emitted element f=
or each window<br>
will be the (endOfWindow - 1). So in your case, in the second windowing ope=
ration (window by 5)<br>
there will be at most one element per window.<br>
<br>
I hope this answers your question.<br>
<span class=3D"m_4387370997278467890HOEnZb"><font color=3D"#888888"><br>
Kostas<br>
</font></span><div class=3D"m_4387370997278467890HOEnZb"><div class=3D"m_43=
87370997278467890h5"><br>
&gt; On Nov 29, 2016, at 7:25 PM, Janardhan Reddy &lt;<a href=3D"mailto:jan=
ardhan.reddy@olacabs.com" target=3D"_blank">janardhan.reddy@olacabs.com</a>=
&gt; wrote:<br>
&gt;<br>
&gt; Hi,<br>
&gt;<br>
&gt; Suppose we have a stream like this.<br>
&gt;<br>
&gt; someStream.timeWindow(Time.min<wbr>utes(15)).apply {<br>
&gt; operation A<br>
&gt;<br>
&gt; }.keyby(&quot;....&quot;).window(Tumblin<wbr>gEventTimeWindows.ofsecon=
ds(5)<wbr>).apply {<br>
&gt;<br>
&gt; operation B<br>
&gt;<br>
&gt; }.keyby(&quot;....&quot;).window(Tumblin<wbr>gEventTimeWindows.ofsecon=
ds(5)<wbr>).apply {<br>
&gt;<br>
&gt; operation C<br>
&gt;<br>
&gt; }<br>
&gt;<br>
&gt; Say operation A emits some elements =3D&gt; it would be emitted every =
15 minutes.<br>
&gt;<br>
&gt; How would be the window behaviour of where operation B takes place if =
operation A takes more than 5 seconds with ingestion Time characteristic.=
=C2=A0 Similarly how would windows behave near operation C if operation B t=
akes more than 5 seconds.<br>
&gt;<br>
&gt;<br>
&gt; Thanks<br>
&gt; Janardhan<br>
<br>
</div></div></blockquote></div><br></div>
</div></blockquote></div><br></div></div></div></div></div></blockquote></d=
iv><br></div>

--94eb2c0859689b19a00542889883--