flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gabriele Di Bernardo <gabriele.diberna...@me.com>
Subject Operations dependencies between values with different key in a ConnectedStreams
Date Fri, 28 Jul 2017 15:08:18 GMT
Hi guys,

I have a question for you. I have an application with two keyed data streams: one for control
and the other one for the data. Each control message represents an operation to be performed
on the data values marked with a certain identifier. I connected the two streams and I process
the data with a CoProcessFunction.

The operations I do are really simple like collecting the MAX or the MEAN value of the last
n seconds. Now, I would like to create more complex operations where the result value of a
key might depend by the result value of another key. To be more clear, I would like to evaluate
expressions like: if {ALL the values of data marked with id 22 in the last 5s} are BIGGER
THAN {The MEAN value of data marked with id 17 in the last 3s}. In this example, I should
defer the evaluation of the expression until I have the MEAN value of the right part of the
expression and check it against ALL the data keyed with key 22 from the last 5 seconds. I’d
like to ask you if something like this in Flink is doable and what is the best way to do that
in your opinion. I am also checking how the CEP library works (can it be helpful?).

Thank you so much in advance.


View raw message