flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From M Singh <mans2si...@yahoo.com>
Subject Re: Flink : CEP processing
Date Tue, 09 Aug 2016 19:45:21 GMT
Hi Sameer:
If we use a within window for event series - 
1. Does it interfere with the default time windows ?2. How does it affect snapshotting ?  3.
If the window is too large are the events stored in a "processor" for the window to expire
?4. Are there any other know limitations and best practices of using CEP with Flink ?
Thanks again for your help.

    On Tuesday, August 9, 2016 11:29 AM, Sameer Wadkar <sameer@axiomine.com> wrote:

 In that case you need to get them into one stream somehow (keyBy a dummy value for example).
There is always some logical key to keyBy on when data is arriving from multiple sources (ex
some portion of the time stamp). 
You are looking for patterns within something (events happening around the same time but arriving
from multiple devices). That something should be the key. That's how I am using it. 

Sent from my iPhone
On Aug 9, 2016, at 1:40 PM, M Singh <mans2singh@yahoo.com> wrote:

Thanks Sameer.
So does that mean that if the events keys are not same we cannot use the CEP pattern match
?  What if events are coming from different sources and need to be correlated ?

    On Tuesday, August 9, 2016 9:40 AM, Sameer W <sameer@axiomine.com> wrote:

You will need to use keyBy operation first to get all the events you need monitored in a pattern
on the same node. Only then can you apply Pattern because it depends on the order of the events
(first, next, followed by). I even had to make sure that the events were correctly sorted
by timestamps to ensure that the first,next and followed by works correctly.
On Tue, Aug 9, 2016 at 12:17 PM, M Singh <mans2singh@yahoo.com> wrote:

Hey Folks:
I have a question about CEP processing in Flink - How does flink processing work when we have
multiple partitions in which the events used in the pattern sequence might be scattered across
multiple partitions on multiple nodes ?
Thanks for your insight.


View raw message