flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Titus Rakkesh <titus.rakk...@gmail.com>
Subject Re: Couldn't figure out - How to do this in Flink? - Pls assist with suggestions
Date Mon, 11 Feb 2019 06:06:37 GMT
Thanks Chesnay. I will try that and let you know.

Thanks.

On Sun, Feb 10, 2019 at 2:31 PM Chesnay Schepler <chesnay@apache.org> wrote:

> You should be able to use a KeyedProcessFunction
> <https://ci.apache.org/projects/flink/flink-docs-master/dev/stream/operators/process_function.html#the-processfunction>for
> that.
> Find matching elements via keyBy() on the first field.
> Aggregate into ValueState, send alert if necessary.
> Upon encountering a new key, setup a timer to remove the entry in 24h.
>
> On 08.02.2019 07:43, Titus Rakkesh wrote:
>
> Dears,
>
> I have a data stream continuously coming,
>
> DataStream<Tuple3<String, Integer, Double>> splitZTuple;
>
> Eg  - (775168263,113182,0.0)
>
> I have to store this for 24 hrs expiry in somewhere (Window or somewhere)
> to check against another stream.
>
> The second stream is
>
> DataStream<Tuple2<String, Double>> splittedVomsTuple which also
> continuously receiving one.
>
> Eg. (775168263,100.0)
>
>
> We need to accumulate the third element in (775168263,113182,*0.0*) in
> the WINDOW (If the corresponding first element match happened with the
> incoming second streams second element 775168263,*100.0*)
>
> While keeping this WINDOW session if any (775168263,113182,*175*) third
> element in the Window Stream exceed a value (Eg >150) we need to call back
> a another rest point to send an alert ------- (775168263,113182,*175*)
> match the criteria. Simply a CEP call back.
>
>
> In Flink how we can do this kind of operations? Or do I need to think
> about any other framework? Please advise.
>
> Thanks...
>
>
>

Mime
View raw message