flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aljoscha Krettek <aljos...@apache.org>
Subject Re: Control triggering on empty window
Date Mon, 25 Apr 2016 13:15:10 GMT
Hi,
I'm sorry but I still think this is not possible. Windows are usually
associated with a key, so if there is no element to which we can assign a
window then there is also no key to which the window would belong.

Cheers,
Aljoscha

On Thu, 21 Apr 2016 at 22:35 Maxim <mfateev@gmail.com> wrote:

> I think the best way to support such a feature is to extend WindowAssigner
> with ability to be called on timer and checkpoint its state the same way it
> is done by the Trigger.
> Such WindowAssigner would be able to create Windows based on time even if
> no event is received.
>
> On Thu, Apr 21, 2016 at 1:57 AM, Aljoscha Krettek <aljoscha@apache.org>
> wrote:
>
>> Hi,
>> I'm afraid this is not possible with our windowing model (expect with
>> hacks using GlobalWindow, as you mentioned). The reason is, that windows
>> only come into existence once there is an element that has a window. Before
>> that, the system has no reference point about what windows there should
>> exist because there is no knowledge about time except when looking at
>> elements.
>>
>> Cheers,
>> Aljoscha
>>
>> On Thu, 21 Apr 2016 at 01:31 Maxim <mfateev@gmail.com> wrote:
>>
>>> I have the following use case:
>>>
>>> Input stream of timestamped "on" and "off" events received out of order.
>>> I need to produce an event with time that system was "on" every 15
>>> minutes. Events should be produced only for intervals that system was "on".
>>>
>>> When 15 minute window has at least one record it is triggered and the
>>> required aggregate is created, but when no event is received within 15
>>> minute period window is not triggered and nothing is produced.
>>>
>>> I understand that it is not feasible to trigger on empty windows when
>>> the set of keys is unbounded. But it would be nice to give the control for
>>> such triggering to a window function. In my case the window function could
>>> enable the empty triggering for the current key when the last event in the
>>> evaluated window is "on" and disable it if is "off".
>>> The strawman API for such feature:
>>>
>>> public void apply(String key, TimeWindow window, Iterable<OnOffEvent> input,
Collector<Aggregate> out) throws Exception {
>>>
>>>     ...
>>>
>>>     RuntimeContext context = this.getRuntimeContext();
>>>
>>>     if (lastEvent.isOn()) {
>>>
>>>        context.enableEmptyWindowTriggering();
>>>
>>>     } else {
>>>
>>>        context.disableEmptyWindowTriggering();
>>>
>>>     }
>>>
>>> }
>>>
>>> I could implement the same logic using global window and custom trigger
>>> and evictor, but it looks like ugly workaround to me.
>>>
>>> Is there any better way to solve this use case?
>>>
>>> Thanks,
>>>
>>> Maxim.
>>>
>>>
>

Mime
View raw message