flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "G.S.Vijay Raajaa" <gsvijayraa...@gmail.com>
Subject Re: Window data retention - Appending to previous data and processing
Date Tue, 27 Jun 2017 20:18:09 GMT
Thanks a lot. It works fine !!

Regards,
Vijay Raajaa GS

On Mon, Jun 26, 2017 at 7:01 PM, Aljoscha Krettek <aljoscha@apache.org>
wrote:

> Hi,
>
> I think you should be able to do this by using:
>
>  * GlobalWindows as your window function
>  * a custom Trigger that fires on every element, sets a timer for your
> cleanup time, and purges when the cleanup timer fires
>  * a ProcessWindowFunction, to so that you always get all the contents of
> the window when processing a window
>
> Best,
> Aljoscha
>
> > On 24. Jun 2017, at 18:37, G.S.Vijay Raajaa <gsvijayraajaa@gmail.com>
> wrote:
> >
> > Hi ,
> >
> > I am trying to implement a flink job which requires a window that keeps
> on adding data to the previous data in the window. The idea is for every
> addition of a new stream of record, the subsequent chain till the sink
> needs to be called. In the next iteration window will have old data + new
> data and the chain is still processed.
> >
> > Iteration 1 - Window ( record_1) -> Proceed with downstream Chaining and
> call sink
> > Iteration 2 - Window(record_1,record_2) -> Proceed with downstream
> Chaining and call sink
> > Iteration n - Window(record_1,record_2,...,record n) -> Proceed with
> downstream Chaining and call sink
> >
> > Finally , clear the window at the configured time of the day.
> >
> > I hope the use case is clear. Looking forward to your thoughts on
> designing the same.
> >
> > Regards,
> > Vijay Raajaa G S
>
>

Mime
View raw message