flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kostas Kloudas <k.klou...@data-artisans.com>
Subject Re: Window Functions and Empty Panes
Date Tue, 18 Apr 2017 21:15:00 GMT
I forgot to say that timers are fault-tolerant. You set them, and Flink takes care of checkpointing
and
restoring them after failure. The flag will also be fault-tolerant as, i suppose, you will
use Flink’s keyed state.

For more info, you can check the ProcessFunction documentation that Konstantin provided.
There, the example uses a value state to hold the counter, you can do sth similar to keep
the flag.
Keep in mind that the state will already be scoped by key so you do not have to worry about
that
either.

Kostas

> On Apr 18, 2017, at 11:11 PM, Kostas Kloudas <k.kloudas@data-artisans.com> wrote:
> 
> No problem! Glad I could help!
> 
> Kostas
> 
>> On Apr 18, 2017, at 11:01 PM, Ryan Conway <ryanmackenzieconway@gmail.com <mailto:ryanmackenzieconway@gmail.com>>
wrote:
>> 
>> Hi Kostas,
>> 
>> Re restarting: I missed that ProcessFunction.OnTimerContext extends ProcessFunction.Context!
Until now my thought was that OnTimerContext did not provide a means of restarting a timer.
>> 
>> Re initial timer, you're right, I'll just need to track a boolean in a state variable
that notes whether or not the timer has been initialized. What I am not confident about is
how to manage timer recovery after a node failure; I imagine it will make sense to not track
this variable. I will do more research and cross that bridge when I get there.
>> 
>> So I think a process function will work just fine, here. Thank you again for your
time, Kostas and Konstantin.
>> 
>> Ryan
>> 
>> On Tue, Apr 18, 2017 at 12:07 PM, Kostas Kloudas <k.kloudas@data-artisans.com
<mailto:k.kloudas@data-artisans.com>> wrote:
>> Hi Ryan,
>> 
>> “A periodic window like this requires the ability to start a timer without an element
and to restart a timer when fired.”
>> 
>> For the second part, i.e. “to restart a timer when fired”, you can re-register
the timer in the onTimer() method (set a 
>> new timer for “now + T"), so that the next one fires after T time units, where
T is your period.
>> 
>> For the first part, where you set the initial timer for a window, this needs to have
a first element right? If not, how
>> do you know the key for which to set the timer? Are all the keys known in advance?
>> 
>> Kostas
>> 
>> 
>> 
>>> On Apr 18, 2017, at 8:35 PM, Ryan Conway <ryanmackenzieconway@gmail.com <mailto:ryanmackenzieconway@gmail.com>>
wrote:
>>> 
>>> A periodic window like this requires the ability to start a timer without an
element and to restart a timer when fired.
>> 
>> 
> 


Mime
View raw message