flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ryan Conway <ryanmackenziecon...@gmail.com>
Subject Re: Window Functions and Empty Panes
Date Tue, 18 Apr 2017 18:35:24 GMT
Hi Konstantin,

Thank you for your response. That is unfortunate as this is a
mission-critical component of our application.

I tried process functions first, but ran into problems because their timers
can only be controlled when processing an element. A periodic window like
this requires the ability to start a timer without an element and to
restart a timer when fired.

At a high level, we're trying to handle the absence of data while Flink
appears to be designed specifically for the presence of data. (I would love
to be corrected if I'm wrong.)

If anyone has any other ideas I am all ears, and if I'm able to find a
solution I'll report it here.

Ryan

On Mon, Apr 17, 2017 at 9:53 PM, Konstantin Knauf <
konstantin.knauf@tngtech.com> wrote:

> Hi Ryan,
>
> afaik there is no way to override this behaviour.
>
> Did you have a look at the ProcessFunction [1]? Depending on your
> specific requirements you might be able to re-implement the desired
> functionality on top of it, but you will have to do all the
> time-handling yourself and you do not have window-scoped state.
>
> Cheers,
>
> Konstantin
>
> [1]
> https://ci.apache.org/projects/flink/flink-docs-release-1.2/dev/stream/
> process_function.html
>
> On 18.04.2017 01:49, Ryan Conway wrote:
> > Hello Flinkers,
> >
> > Is there a means of configuring a windowed stream such that its window
> > function is invoked for empty panes?
> >
> > Looking at the source of WindowOperator in Flink 1.2.0 (line 473), I see
> > that the user's window function is invoked only when the pane isn't
> > empty. I am hoping for a means of overriding this behavior.
> >
> > Alternatively... is there another mean of creating a stream with a
> > periodic output based on each period's inputs and some stored state?
> >
> > Thank you,
> > Ryan
>
> --
> Konstantin Knauf * konstantin.knauf@tngtech.com * +49-174-3413182
> TNG Technology Consulting GmbH, Betastr. 13a, 85774 Unterföhring
> Geschäftsführer: Henrik Klagges, Christoph Stock, Dr. Robert Dahlke
> Sitz: Unterföhring * Amtsgericht München * HRB 135082
>
>

Mime
View raw message