flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kostas Kloudas <k.klou...@data-artisans.com>
Subject Re: About Sliding window
Date Wed, 12 Oct 2016 14:32:03 GMT
Hello,

So in your previous figure (yesterday) when e3 arrives, also e2 should be included in the
result, right?

In this case, I think that what you need is a Session window with gap equal to your event
aging duration and
an evictor that evicts the elements that lag behind more than the gap duration.

The latter, the evictor that I am describing, is not currently supported in Flink but there
is an ongoing 
discussion in the dev mailing list about it. So it is worth having a look there and participate
in the discussion.

I also loop in Aljoscha in the discussion, in case he has another solution that you can deploy
right-away.

Thanks,
Kostas

> On Oct 12, 2016, at 3:36 PM, Zhangrucong <zhangrucong@huawei.com> wrote:
> 
> Hi Kostas:
>             It doesn’t matter. Can you see the picture? My user case is:
>  
> 1、The events are coming according to the following order
> <image004.png>
> At 9:01 e1 is coming
> At 9:02 e2 is coming
> At 9:06  e3 is coming
> At 9:08   e4 is coming
>  
> The time is system time.
>  
> 2、And  event aging time is 5 minutes.
>  
> 3、
>        At 9:01 e1 is coming, aged nothing, store e1,we count e1 and send the result.
>        At 9:02 e2 is coming,  aged nothing, store e2,  We count e1 and e2. and send the
result.
>       At 9:06  e3 is coming,  aged e1,  store e3, we count e2 and e3, and send the result.
>      At 9:08   e4 is coming,  aged e2,  store e4, we count e3 and e4, and send the result.
>  
>  
> I think I need a certain duration window.
>  
> Thank you very much!
> 发件人: Kostas Kloudas [mailto:k.kloudas@data-artisans.com] 
> 发送时间: 2016年10月12日 21:11
> 收件人: Zhangrucong
> 抄送: user@flink.apache.org
> 主题: Re: About Sliding window
>  
> Hello again,
>  
> Sorry for the delay but I cannot really understand your use case.
> Could you explain a bit more what do you mean by “out-of-date” event and “aging”
an event?
>  
> Also your windows are of a certain duration or global?
>  
> Thanks,
> Kostas
>  
> On Oct 11, 2016, at 3:04 PM, Zhangrucong <zhangrucong@huawei.com <mailto:zhangrucong@huawei.com>>
wrote:
>  
> Hi Kostas:
> Thank you for your rapid response!
>  
> My use-case is that :
> For every incoming event, we want to age the out-of-date event , count the event in window
and send the result.
>  
> For example:
> The events are coming as flowing:
> <image002.png>
>  
> We want flowing result:
> <image004.png>
>  
>  
> By the way, In StreamSQL API, in FILP11, It will realize row window. It seems that the
function of Slide Event-time row-window suits my use-case. Does data stream API  support row
window?
>  
> Thanks !
>  
> 发件人: Kostas Kloudas [mailto:k.kloudas@data-artisans.com <mailto:k.kloudas@data-artisans.com>]

> 发送时间: 2016年10月11日 19:38
> 收件人: user@flink.apache.org <mailto:user@flink.apache.org>
> 主题: Re: About Sliding window
>  
> Hi Zhangrucong,
>  
> Sliding windows only support time-based slide. 
> So your use-case is not supported out-of-the-box.
>  
> But, if you describe a bit more what you want to do, 
> we may be able to find a way together to do your job using 
> the currently offered functionality.
>  
> Kostas
>  
> On Oct 11, 2016, at 1:20 PM, Zhangrucong <zhangrucong@huawei.com <mailto:zhangrucong@huawei.com>>
wrote:
>  
> Hello everyone:
>   Now, I am want to use DataStream sliding window API. I look at the API and I have a
question, dose the sliding time window support sliding by every incoming event?
>  
> Thanks in advance!
>  
> <oledata.mso>
>  
> <oledata.mso>


Mime
View raw message