flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kostas Kloudas <k.klou...@data-artisans.com>
Subject Re: Query regarding tumbling event time windows with ingestion time
Date Wed, 30 Nov 2016 22:37:23 GMT
No problem! 

Kostas

> On Nov 30, 2016, at 7:08 PM, Janardhan Reddy <janardhan.reddy@olacabs.com> wrote:
> 
> That makes it clear.
> 
> Thanks
> 
> On Wed, Nov 30, 2016 at 10:15 PM, Kostas Kloudas <k.kloudas@data-artisans.com <mailto:k.kloudas@data-artisans.com>>
wrote:
> Hello,
> 
> If you have events:
> 
> A@10m, B@14m, Watermark@15m 
> C@16m D@25m, Watermark@30m
> 
> then the result will be:
> 
> [A,B]@14.59
> [C,D]@29.59
> 
> This means that for the next windowing, you will have the elements [A,B]@14.59 and [C,D]@29.59
> to window in windows of 5min. 
> 
> Given that they are 15min apart, this means that you will have:
> 
> [A,B]@14.59 in one window and [C,D]@29.59
> 
> This is what I meant by at most one element per window, because window 0 to 5 min, 5
to 10 and 
> 15 to 20 and 20 to 25 will be empty.
> 
> Does this make it clearer?
> 
> Kostas
> 
>> On Nov 30, 2016, at 11:48 AM, Janardhan Reddy <janardhan.reddy@olacabs.com <mailto:janardhan.reddy@olacabs.com>>
wrote:
>> 
>> HI
>> i didn't get it , can you please clarify with an example in case each of operation
A and B emit multiple elements.
>> 
>> On Wed, Nov 30, 2016 at 3:34 PM, Kostas Kloudas <k.kloudas@data-artisans.com <mailto:k.kloudas@data-artisans.com>>
wrote:
>> Hi Janardhan,
>> 
>> After the first windowing operation, the timestamp of the emitted element for each
window
>> will be the (endOfWindow - 1). So in your case, in the second windowing operation
(window by 5)
>> there will be at most one element per window.
>> 
>> I hope this answers your question.
>> 
>> Kostas
>> 
>> > On Nov 29, 2016, at 7:25 PM, Janardhan Reddy <janardhan.reddy@olacabs.com
<mailto:janardhan.reddy@olacabs.com>> wrote:
>> >
>> > Hi,
>> >
>> > Suppose we have a stream like this.
>> >
>> > someStream.timeWindow(Time.minutes(15)).apply {
>> > operation A
>> >
>> > }.keyby("....").window(TumblingEventTimeWindows.ofseconds(5)).apply {
>> >
>> > operation B
>> >
>> > }.keyby("....").window(TumblingEventTimeWindows.ofseconds(5)).apply {
>> >
>> > operation C
>> >
>> > }
>> >
>> > Say operation A emits some elements => it would be emitted every 15 minutes.
>> >
>> > How would be the window behaviour of where operation B takes place if operation
A takes more than 5 seconds with ingestion Time characteristic.  Similarly how would windows
behave near operation C if operation B takes more than 5 seconds.
>> >
>> >
>> > Thanks
>> > Janardhan
>> 
>> 
> 
> 


Mime
View raw message