flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Luis Mariano Guerra <mari...@event-fabric.com>
Subject Re: emit partial state in window (streaming)
Date Thu, 03 Nov 2016 18:08:36 GMT
On Thu, Nov 3, 2016 at 7:05 PM, Kostas Kloudas <k.kloudas@data-artisans.com>
wrote:

> Hi Luis,
>
> You cannot have event-time early firings on both chained window operators.
> The reason is that each early result from the first window operator will
> have a timestamp equal to window.maxTimestamp-1.
> So in the second windowing operator, they will be buffered until the
> watermark signaling the end of the window arrives.
>

so, how do I get windowAll and partial results? do I have to remove the
partial calculations and do it all in one node/thread? is there another way?


>
> Now for the second point, I think that what you have understood is correct.
> The "ctx.registerEventTimeTimer(window.getEnd())” registers a timer to
> call the onEventTime().
>

then what else calls onEventTime? because if a register the event time
timer inside of it something else calls it.



>
> Cheers,
> Kostas
>
>
> > On Nov 3, 2016, at 3:29 PM, Luis Mariano Guerra <
> mariano@event-fabric.com> wrote:
> >
> > On Thu, Nov 3, 2016 at 2:06 PM, Kostas Kloudas <
> k.kloudas@data-artisans.com> wrote:
> > Hi Luis,
> >
> > Can you try to comment the whole final windowing and see if this is
> works?
> > This includes the following lines:
> >
> >   .windowAll(TumblingEventTimeWindows.of(Time.of(windowTime, timeUnit)))
> >   .trigger(new PartialWindowTrigger<>(partialWindowTime, timeUnit,
> windowTime, timeUnit))
> >   .apply(creator.create(), windowAllFold, windowAllMerge);
> >
> >
> > commenting it emits on fire, how do I make the trigger "go thorough" the
> windowAll, or if not possible, how can I join the substreams in one stream
> and respect the trigger?
> >
> > An additional note is that I would go for registering an event time
> timer at the onEventTime
> > instead of checking the timestamp on the onElement(). This is because
> with your implementation,
> > in order to fire a computation, you always have to wait for an element
> outside the partial window interval to arrive.
> >
> > then I think I understood the purpose of registering the event time
> timer wrong, isn't "ctx.registerEventTimeTimer(window.getEnd())" called
> to register a timer to call onEventTime?
> >
> >
> > Cheers,
> > Kostas
> >
> >> On Nov 3, 2016, at 11:31 AM, Luis Mariano Guerra <
> mariano@event-fabric.com> wrote:
> >>
> >>   .windowAll(TumblingEventTimeWindows.of(Time.of(windowTime,
> timeUnit)))
> >>                 //.trigger(new PartialWindowTrigger<>(partialWindowTime,
> timeUnit, windowTime, timeUnit))
> >>                 .apply(creator.create(), windowAllFold, windowAllMerge);
> >
> >
>
>

Mime
View raw message