flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jaya sai <jayasai...@gmail.com>
Subject question for handling db data
Date Thu, 25 Jul 2019 22:48:27 GMT
Hello,

I have a question on using flink, we have a small data set which does not
change often but have another data set which we need to compare with it and
it has lots of data

let say I have two collections geofence and locations in mongodb. Geofence
collection does not change often and relatively small, but we have location
data coming in at high amounts from clients and we need to calculate the
goefence entry exits based on geofence and location data point.
For calculating the entry and exit we were thinking of using flink CEP. But
our problem is sometimes geofence data changes and we need to update the in
memory store of the flink somehow

we were thinking of bootstrapping the memory of flink processor by loading
data on initial start and subscribe to kafaka topic to listen for geofence
changes and re-pull the data
Is this a valid approach ?

Thank you,

Mime
View raw message