flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chesnay Schepler <ches...@apache.org>
Subject Re: Couldn't figure out - How to do this in Flink? - Pls assist with suggestions
Date Sun, 10 Feb 2019 09:01:54 GMT
You should be able to use a KeyedProcessFunction 
<https://ci.apache.org/projects/flink/flink-docs-master/dev/stream/operators/process_function.html#the-processfunction>for

that.
Find matching elements via keyBy() on the first field.
Aggregate into ValueState, send alert if necessary.
Upon encountering a new key, setup a timer to remove the entry in 24h.

On 08.02.2019 07:43, Titus Rakkesh wrote:
>
> Dears,
>
> I have a data stream continuously coming,
>
> DataStream<Tuple3<String, Integer, Double>> splitZTuple;
>
> Eg  - (775168263,113182,0.0)
>
> I have to store this for 24 hrs expiry in somewhere (Window or 
> somewhere) to check against another stream.
>
> The second stream is
>
> DataStream<Tuple2<String, Double>> splittedVomsTuple which also 
> continuously receiving one.
>
> Eg. (775168263,100.0)
>
>
> We need to accumulate the third element in (775168263,113182,*/0.0/*) 
> in the WINDOW (If the corresponding first element match happened with 
> the incoming second streams second element 775168263,*/100.0/*)
>
> While keeping this WINDOW session if any (775168263,113182,*/175/*) 
> third element in the Window Stream exceed a value (Eg >150) we need to 
> call back a another rest point to send an alert ------- 
> (775168263,113182,*/175/*) match the criteria. Simply a CEP call back.
>
>
> In Flink how we can do this kind of operations? Or do I need to think 
> about any other framework? Please advise.
>
> Thanks...
>


Mime
View raw message