flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Janardhan Reddy <janardhan.re...@olacabs.com>
Subject Re: Query regarding tumbling event time windows with ingestion time
Date Wed, 30 Nov 2016 18:08:06 GMT
That makes it clear.

Thanks

On Wed, Nov 30, 2016 at 10:15 PM, Kostas Kloudas <
k.kloudas@data-artisans.com> wrote:

> Hello,
>
> If you have events:
>
> A@10m, B@14m, Watermark@15m
> C@16m D@25m, Watermark@30m
>
> then the result will be:
>
> [A,B]@14.59
> [C,D]@29.59
>
> This means that for the next windowing, you will have the elements
> [A,B]@14.59 and [C,D]@29.59
> to window in windows of 5min.
>
> Given that they are 15min apart, this means that you will have:
>
> [A,B]@14.59 in one window and [C,D]@29.59
>
> This is what I meant by at most one element per window, because window 0
> to 5 min, 5 to 10 and
> 15 to 20 and 20 to 25 will be empty.
>
> Does this make it clearer?
>
> Kostas
>
> On Nov 30, 2016, at 11:48 AM, Janardhan Reddy <janardhan.reddy@olacabs.com>
> wrote:
>
> HI
> i didn't get it , can you please clarify with an example in case each of
> operation A and B emit multiple elements.
>
> On Wed, Nov 30, 2016 at 3:34 PM, Kostas Kloudas <
> k.kloudas@data-artisans.com> wrote:
>
>> Hi Janardhan,
>>
>> After the first windowing operation, the timestamp of the emitted element
>> for each window
>> will be the (endOfWindow - 1). So in your case, in the second windowing
>> operation (window by 5)
>> there will be at most one element per window.
>>
>> I hope this answers your question.
>>
>> Kostas
>>
>> > On Nov 29, 2016, at 7:25 PM, Janardhan Reddy <
>> janardhan.reddy@olacabs.com> wrote:
>> >
>> > Hi,
>> >
>> > Suppose we have a stream like this.
>> >
>> > someStream.timeWindow(Time.minutes(15)).apply {
>> > operation A
>> >
>> > }.keyby("....").window(TumblingEventTimeWindows.ofseconds(5)).apply {
>> >
>> > operation B
>> >
>> > }.keyby("....").window(TumblingEventTimeWindows.ofseconds(5)).apply {
>> >
>> > operation C
>> >
>> > }
>> >
>> > Say operation A emits some elements => it would be emitted every 15
>> minutes.
>> >
>> > How would be the window behaviour of where operation B takes place if
>> operation A takes more than 5 seconds with ingestion Time characteristic.
>> Similarly how would windows behave near operation C if operation B takes
>> more than 5 seconds.
>> >
>> >
>> > Thanks
>> > Janardhan
>>
>>
>
>

Mime
View raw message