flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "dyana.rose@salecycle.com"<dyana.r...@salecycle.com>
Subject Re: Dynamic SessionWindow gaps
Date Fri, 12 Jan 2018 10:09:20 GMT

I've put up some implementation information and a link to the working branch for comment while
I get the documentation written up: 

https://issues.apache.org/jira/browse/FLINK-8384

Thanks,
Dyana

On 2018-01-02 13:54, jincheng sun <sunjincheng121@gmail.com> wrote: 
> Hi, Dyana dynamic gap is very useful feature, Agree with the suggestion of
> Aljoscha, looking forward the PR.
> 
> Best, Jincheng
> 
> 2018-01-02 21:45 GMT+08:00 Aljoscha Krettek <aljoscha@apache.org>:
> 
> > Hi,
> >
> > This sounds good! Could you please open a Jira Issue for this and then
> > open a PR against that Jira issue?
> >
> > Btw, I would suggest to implement this as a new type of assigner,
> > something like DynamicSessionWindows.
> >
> > Best,
> > Aljoscha
> >
> > > On 29. Dec 2017, at 20:54, Dyana Rose <dyana.rose@salecycle.com> wrote:
> > >
> > > I have a use case for non-static Session Window gaps.
> > >
> > > For example, given a stream of IoT events, each device type could have a
> > > different gap, and that gap could change while sessions are in flight.
> > >
> > > I didn't want to have to run a stream processor for each potential gap
> > > length, not to mention the headache of dealing with changing gaps, so
> > I've
> > > implemented a version of SessionWindows that has one major change; in the
> > > assignWindows method it passes the element to a method to extract the
> > > correct sessionTimeout. (current Flink method for reference:
> > > https://github.com/apache/flink/blob/master/flink-
> > streaming-java/src/main/java/org/apache/flink/streaming/
> > api/windowing/assigners/EventTimeSessionWindows.java#L59
> > > )
> > >
> > > Preliminary tests show this working as required and I can't be the only
> > > person with this type of use case for session windows.
> > >
> > > Will an issue and PR to add this functionality to the SessionWindow
> > classes
> > > be welcome?
> > >
> > > Dyana
> >
> >
> 

Mime
View raw message