flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rong Rong <walter...@gmail.com>
Subject Re: Propagating delta from window upon trigger
Date Wed, 22 May 2019 17:39:26 GMT
Hi Miki,

Upon trigger, window will be fired with all of its content in its state
invoking the "emitWindowContent" method. which will further invoke the
window function you define.
Thus if your goal is to only emit the delta, one thing is to do so in your
window function. One challenge you might face is how to make your window
function stateful, you might want to checkout this document[1] for more
details.

If your goal is to limit the number of content stored in the window state
to only the delta. you can also look at incremental aggregation state [2]
of the window, but you will have to design your window function in such a
way that it can take delta into account.

--
Rong

[1]
https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/operators/windows.html#processwindowfunction
[2]
https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/operators/windows.html#processwindowfunction-with-incremental-aggregation


On Sun, May 19, 2019 at 8:00 PM Congxian Qiu <qcx978132955@gmail.com> wrote:

> Hi, Nikhil
> Window will emit all state to downstream. Can you clear the state while
> triggering?
>
> Nikhil Goyal <nownikhil@gmail.com>于2019年5月19日 周日01:03写道:
>
>> I have window of 1 hour and trigger of 5 min. I want to know if every 5
>> min Flink is writing the entire window or only the keys which changed.
>>
>> On Sat, May 18, 2019, 9:40 AM miki haiat <miko5054@gmail.com> wrote:
>>
>>> Can you elaborate more what  is you use case ?
>>>
>>>
>>> On Sat, May 18, 2019 at 12:47 AM Nikhil Goyal <nownikhil@gmail.com>
>>> wrote:
>>>
>>>> Hi guys,
>>>>
>>>> Is there a way in Flink to only propagate the changes which happened in
>>>> the window's state rather than dumbing the contents of the window again and
>>>> again upon trigger?
>>>>
>>>> Thanks
>>>> Nikhil
>>>>
>>> --
> Best,
> Congxian
>

Mime
View raw message