flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kostas Kloudas <k.klou...@data-artisans.com>
Subject Re: Multiple CEP Patterns
Date Fri, 28 Apr 2017 07:48:28 GMT
Sorry for the quick followup, but another question, in case the JIRA I sent you 
is not what affects your job, do your patterns have a timeout (the within() clause) ?

If not, then also other parts of the system (e.g. the internal state of your NFA)
may grow indefinitely.

Kostas

> On Apr 28, 2017, at 9:44 AM, Kostas Kloudas <k.kloudas@data-artisans.com> wrote:
> 
> Hi!
> 
> I suppose that by memory errors you mean you run out of memory, right?
> 
> Are you using Flink 1.2 or the current master (upcoming Flink 1.3).
> The reason I am asking is because Flink 1.2 suffered from this
> https://issues.apache.org/jira/browse/FLINK-5174 <https://issues.apache.org/jira/browse/FLINK-5174>
> which is now fixed in Flink 1.3, and you are more than welcome to 
> try it out, to also help us with testing the new features.
> 
> Now if this is not the case, could you share a bit more details 
> about your program?
> 
> You do a CEP.pattern(input, pattern_x) for each of your patterns? 
> (input is your input stream and x the index of each pattern)
> 
> Thanks,
> Kostas
>  
>> On Apr 27, 2017, at 8:59 PM, mclendenin <marcuscbey@gmail.com <mailto:marcuscbey@gmail.com>>
wrote:
>> 
>> I'm trying to run multiple independent CEP patterns. They're basic patterns,
>> just one input followed by another and my flink job runs fine when just
>> using 1 pattern. If i try to scale this up to add multiple CEP patterns, 200
>> for example, I start getting memory errors on my cluster. I can definitely
>> add more memory, but I want to know if there is an accepted way to run
>> multiple patterns.
>> 
>> Currently I am just doing a flatselect on the output of CEP.pattern and then
>> making a list of all these inputs and putting a sink on all of the streams.
>> What this is doing is creating a different stream for each pattern, which is
>> causing the output of the Logical Plan in the UI to be too big to even see.
>> Does anybody know of any better way to do this?
>> 
>> 
>> 
>> --
>> View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Multiple-CEP-Patterns-tp12871.html
<http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Multiple-CEP-Patterns-tp12871.html>
>> Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com
<http://nabble.com/>.
> 


Mime
View raw message