Mailing-List: contact user-help@flink.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@flink.apache.org
MIME-Version: 1.0
In-Reply-To: <371215EB-46E3-4AEC-9A4C-BE2A258129F7@data-artisans.com>
References: <CABYPoJm=C6p7j27Sz1wCQjQBSd4ggD_zuBjvPfsSFHu4Cs1PzA@mail.gmail.com>
 <371215EB-46E3-4AEC-9A4C-BE2A258129F7@data-artisans.com>
From: Yassin Marzouki <yassmarzou@gmail.com>
Date: Wed, 27 Jul 2016 14:04:24 +0200
Message-ID: <CABYPoJ=ft62YNzYKwrpMfNLJGj8=cVuWJ7Bg5diaEUQrDEA+zw@mail.gmail.com>
Subject: Re: If I chain two windows, what event-time would the second window have?
To: user@flink.apache.org
Content-Type: multipart/alternative; boundary=089e014934ace0d05d05389cd383
archived-at: Wed, 27 Jul 2016 12:05:10 -0000

--089e014934ace0d05d05389cd383
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Hi Kostas,

Thank you very much for the explanation.

Best,
Yassine

On Wed, Jul 27, 2016 at 1:09 PM, Kostas Kloudas <k.kloudas@data-artisans.co=
m
> wrote:

> Hi Yassine,
>
> When the WindowFunction is applied to the content of a window, the
> timestamp of the resulting record
> is the window.maxTimestamp, which is the endOfWindow-1.
>
> You can imaging if you have a Tumbling window from 0 to 2000, the result
> will have a timestamp of 1999.
> Window boundaries are closed in the start and open at the end timestamp,
> or [start, end).
>
> If you want to play around, I would suggest checking out the tests in the
> WindowOperatorTest class.
>
> There you can do experiments and figure out how Flink=E2=80=99s windowOpe=
rator
> works internally and what is the
> interplay between windowAssingers, triggers, and the windowOperator.
>
> Hope this helps,
> Kostas
>
> On Jul 27, 2016, at 8:41 AM, Yassin Marzouki <yassmarzou@gmail.com> wrote=
:
>
> Hi all,
>
> Say I assign timestamps to a stream and then apply a transformation like
> this:
>
>
> stream.keyBy(0).timeWindow(Time.hours(5)).reduce(count).timeWindowAll(Tim=
e.days(1)).apply(transformation)
>
> Now, when the first window is applied, events are aggregated based on
> their timestamps, but I don't understand what timestamp will be assigned =
to
> the aggregated result of the reduce operation for the second window to
> process it. Could you please explain it? Thank you.
>
> Best,
> Yassine
>
>
>

--089e014934ace0d05d05389cd383
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><font face=3D"verdana, sans-serif">Hi Kostas,</font><div><=
font face=3D"verdana, sans-serif"><br></font></div><div><font face=3D"verda=
na, sans-serif">Thank you very much for the explanation.</font></div><div><=
font face=3D"verdana, sans-serif"><br></font></div><div><font face=3D"verda=
na, sans-serif">Best,</font></div><div><font face=3D"verdana, sans-serif">Y=
assine</font><br></div></div><div class=3D"gmail_extra"><br><div class=3D"g=
mail_quote">On Wed, Jul 27, 2016 at 1:09 PM, Kostas Kloudas <span dir=3D"lt=
r">&lt;<a href=3D"mailto:k.kloudas@data-artisans.com" target=3D"_blank">k.k=
loudas@data-artisans.com</a>&gt;</span> wrote:<br><blockquote class=3D"gmai=
l_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left=
:1ex"><div style=3D"word-wrap:break-word">Hi Yassine,<div><br></div><div>Wh=
en the WindowFunction is applied to the content of a window, the timestamp =
of the resulting record</div><div>is the window.maxTimestamp, which is the =
endOfWindow-1.</div><div><br></div><div>You can imaging if you have a Tumbl=
ing window from 0 to 2000, the result will have a timestamp of 1999.</div><=
div>Window boundaries are closed in the start and open at the end timestamp=
, or [start, end).</div><div><br></div><div>If you want to play around, I w=
ould suggest checking out the tests in the WindowOperatorTest class.</div><=
div><br></div><div>There you can do experiments and figure out how Flink=E2=
=80=99s windowOperator works internally and what is the=C2=A0</div><div>int=
erplay between windowAssingers, triggers, and the windowOperator.</div><div=
><br></div><div>Hope this helps,</div><div>Kostas</div><div><div class=3D"h=
5"><div><br></div><div><div><blockquote type=3D"cite"><div>On Jul 27, 2016,=
 at 8:41 AM, Yassin Marzouki &lt;<a href=3D"mailto:yassmarzou@gmail.com" ta=
rget=3D"_blank">yassmarzou@gmail.com</a>&gt; wrote:</div><br><div><div dir=
=3D"ltr"><font face=3D"verdana, sans-serif">Hi all,</font><div><font face=
=3D"verdana, sans-serif"><br></font></div><div><font face=3D"verdana, sans-=
serif">Say I assign timestamps to a stream and then apply a transformation =
like this:</font></div><div><font face=3D"verdana, sans-serif"><br></font><=
/div><div><div><font face=3D"verdana, sans-serif">stream.keyBy(0).timeWindo=
w(Time.hours(5)).reduce(count).timeWindowAll(Time.days(1)).apply(transforma=
tion)</font></div></div><div><font face=3D"verdana, sans-serif"><br></font>=
</div><div><font face=3D"verdana, sans-serif">Now, when the first window is=
 applied, events are aggregated based on their timestamps, but I don&#39;t =
understand what timestamp will be assigned to the aggregated result of the =
reduce operation for the second window to process it.=C2=A0<span style=3D"f=
ont-size:13.44px">Could you please explain it? Thank you.</span></font></di=
v><div><span style=3D"font-size:13.44px"><font face=3D"verdana, sans-serif"=
><br></font></span></div><div><span style=3D"font-size:13.44px"><font face=
=3D"verdana, sans-serif">Best,</font></span></div><div><span style=3D"font-=
size:13.44px"><font face=3D"verdana, sans-serif">Yassine</font></span></div=
></div>
</div></blockquote></div><br></div></div></div></div></blockquote></div><br=
></div>

--089e014934ace0d05d05389cd383--