flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Spico Florin <spicoflo...@gmail.com>
Subject Apache Flink CEP how to detect if event did not occur within x seconds?
Date Wed, 12 Dec 2018 15:24:57 GMT
Hello!
I'm using the same questions as in this stackoverflow post
https://stackoverflow.com/questions/50356981/apache-flink-cep-how-to-detect-if-event-did-not-occur-within-x-seconds?rq=1,
due to fact that I need the same functionality.

My use case is to detect when an device is disconnected from a server (for
example).
>From SNMP I'm sending the data to Kafka then  to Flink CEP and check
whether I have the established connection within X seconds.
If the window of X seconds timedout, then I can consider that the device is
disconnected.
The issue is that, when the device is disconnected, SNMP does't send
anymore data about that port that was used. Thus no event is sent to CEP.
Only when the device is connected again then, the disconnected event is
triggered.

The same functionality was requested in this post too:
https://stackoverflow.com/questions/51399973/flink-time-window-based-on-event-time-output-nothing
.

I checked the status of the jira ticket regarding this feature:

https://issues.apache.org/jira/browse/FLINK-7384

but unfortunately, is still open.

In the official documentation,
https://ci.apache.org/projects/flink/flink-docs-stable/dev/event_time.html,
in the paragraph related with Idling sources, somehow the situation
described resembles what we need:
"Currently, with pure event time watermarks generators, watermarks can not
progress if there are no elements to be processed. That means in case of
gap in the
incoming data, event time will not progress and for example the window
operator will not be triggered and thus existing windows will not be able
to produce any output data.
To circumvent this one can use periodic watermark assigners that don’t only
assign based on element timestamps. An example solution could be
an assigner that switches to using current processing time as the time
basis after not observing new events for a while."

Unfortunately, no example of how the mentioned assigner looks like. A full
example will help us to check whether the mentioned solution works or not.

Therefore, can you please share your thoughts about this feature?
Is feasible? If yes,please provide the Flink version that solve it and the
example.
Is still a limitation and there is no plan to solve it in near future?

I look forward for your answers.

Best regards,
Florin

Mime
View raw message