flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bruno Aranda <bara...@apache.org>
Subject Any good ideas for online/offline detection of devices that send events?
Date Fri, 03 Mar 2017 17:27:39 GMT
Hi all,

We are trying to write an online/offline detector for devices that keep
streaming data through Flink. We know how often roughly to expect events
from those devices and we want to be able to detect when any of them stops
(goes offline) or starts again (comes back online) sending events through
the pipeline. For instance, if 5 minutes have passed since the last event
of a device, we would fire an event to indicate that the device is offline.

The data from the devices comes through Kafka, with their own event time.
The devices events are in order in the partitions and each devices goes to
a specific partition, so in theory, we should not have out of order when
looking at one partition.

We are assuming a good way to do this is by using sliding windows that are
big enough, so we can see the relevant gap before/after the events for each
specific device.

We were wondering if there are other ideas on how to solve this.

Many thanks!

Bruno

Mime
View raw message