flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gyula Fóra <gyf...@apache.org>
Subject Re: Why TimerService interface in ProcessFunction doesn't have deleteEventTimeTimer
Date Fri, 21 Apr 2017 19:23:15 GMT

I thought I would drop my opinion here maybe it is relevant.

We have used the Flink internal timer implementation in many of our
production applications, this supports the Timer deletion but the deletion
actually turned out to be a huge performance bottleneck because of the bad
deletion performance of the Priority queue.

In many of our cases deletion could have been avoided by some more clever
registration/firing logic and we also ended up completely avoiding deletion
and instead using "tombstone" markers by setting a flag in the state which
timers not to fire when they actually trigger.


Aljoscha Krettek <aljoscha@apache.org> ezt írta (időpont: 2017. ápr. 21.,
P, 14:47):

> Hi,
> the reasoning behind the limited user facing API was that we were (are)
> not sure whether we would be able to support efficient deletion of timers
> for different ways of storing timers.
> @Stephan, If I remember correctly you were the strongest advocate for not
> allowing timer deletion. What’s your thinking on this? There was also a
> quick discussion on https://issues.apache.org/jira/browse/FLINK-3089 where
> Xiaogang explained that the (new, not merged) RocksDB based timers would
> have efficient timer deletion.
> Best,
> Aljoscha
> On 20. Apr 2017, at 11:56, Jagadish Bihani <jagadish@helpshift.com> wrote:
> Hi
> I am working on a use case where I want to start a timer for a given event
> type and when that timer expires it will perform certain action. This can
> be done using Process Function.
> But I also want to cancel scheduled timer in case of some other types of
> events. I also checked the implementation of HeapInternalTimerService which
> implements InternalTimerService interface has those implementations
> already. Also SimpleTimerService which overrides TimerService also uses
> InternalTimerService and simply passes VoidNamespace.INSTANCE.
> So in a way we are using InternalTimerService interface's implementations
> everywhere.
> So what is the reason that ProcessFunction.Context uses TimerService? Any
> reason 'deleteEventTimeTimer' is not exposed to users? If I want to use the
> deleteEvent functionality how should I go about it?
> --
> Thanks and Regards,
> Jagadish Bihani

View raw message