flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aljoscha Krettek <aljos...@apache.org>
Subject Re: Custom Barrier?
Date Tue, 14 Jun 2016 08:32:58 GMT
would these super-structure events occur per key? If yes, then I think you
can process this using the currently available windowing mechanism by
writing a custom WindowAssigner and Trigger. This, of course, assumes that
the events arrive in-order, i.e. if A-End arrives before A-Start or if
elements that should fall inside the A window arrive after A-End then I
don't see an easy way to do it.

Let me know if you need to know more about assigners/triggers.


On Mon, 13 Jun 2016 at 16:29 Paul Wilson <paulalexwilson@gmail.com> wrote:

> Hi,
> I've been evaluating Flink and wondering if it was possible to define a
> window that is based on characteristics of the data (data driven) but not
> contained in the data stream directly.
> Consider 'nested events' where lower level events belong to a wider event
> where the wider event serves only to define a boundary (or window) over the
> lower level events. I was wondering if there was some way to communicate
> this super-structure in the stream somehow?
> I know that Flink users 'barriers' to define snapshot boundaries, but it
> might it be possible to communicate a 'window end' in a similar fashion?
> I guess I could attach an additional value to each event using a stateful
> map function and then define the window on that?
> e.g. A-Start, 1, 2, 3, A-End, B-Start, 1, 2, 3, B-End
> Regards,
> Paul

View raw message