flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Koch <ogd...@googlemail.com>
Subject Re: Listening to timed-out patterns in Flink CEP
Date Sun, 09 Oct 2016 12:51:09 GMT
Hello,

Thank you for the explanation as well as the link to the other post.
Interesting to learn about some of the open JIRAs.

Indeed, I was not using event time, but processing time. However, even when
using event time I only get notified of timeouts upon subsequent events.

The link <http://pastebin.com/x4m3RHQz> contains an example where I read
<key> <value> from a socket, wrap this in a custom "event" with timestamp,
key the resultant stream by <key> and attempt to detect <key> instances no
further than 3 seconds apart using CEP.

Apart from the fact that results are only printed when I close the socket
(normal?) I don't observe any change in behaviour

So event-time/watermarks or not: SOME event has to occur for the timeout to
be triggered.

FLINK-3320 <https://issues.apache.org/jira/browse/FLINK-3320> (CEP "not"
operator) does not address this because again, how would the "not match" be
triggered if no event at all occurs?

On Sat, Oct 8, 2016 at 12:50 AM, <lgfmt@yahoo.com> wrote:

> The following is a better link:
>
> http://mail-archives.apache.org/mod_mbox/flink-user/201609.mbox/%3CCAC27z%
> 3DOTtv7USYUm82bE43-DkoGfVC4UAWD6uQwwRgTsE5be8g%40mail.gmail.com%3E
>
>
> - LF
>
>
>
>
> ------------------------------
> *From:* "lgfmt@yahoo.com" <lgfmt@yahoo.com>
> *To:* "user@flink.apache.org" <user@flink.apache.org>
> *Sent:* Friday, October 7, 2016 3:36 PM
>
> *Subject:* Re: Listening to timed-out patterns in Flink CEP
>
> Isn't the upcoming CEP negation (absence of an event) feature solve this
> issue?
>
> See this discussion thread:
> http://mail-archives.apache.org/mod_mbox/flink-user/
> 201609.mbox/%3CCAC27z%3DOD%2BTq8twBw_1YKni5sWAU3g1S9WDpJw0DUwgiG9YX
> 9Fg%40mail.gmail.com%3E
>
>
>
> //  Atul
>
>
> ------------------------------
> *From:* Till Rohrmann <trohrmann@apache.org>
> *To:* user@flink.apache.org
> *Sent:* Friday, October 7, 2016 12:58 AM
> *Subject:* Re: Listening to timed-out patterns in Flink CEP
>
> Hi David,
>
> in case of event time, the timeout will be detected when the first
> watermark exceeding the timeout value is received. Thus, it depends a
> little bit how you generate watermarks (e.g. periodically, watermark per
> event).
>
> In case of processing time, the time is only updated whenever a new
> element arrives. Thus, if you have an element arriving 4 seconds after
> Event A, it should detect the timeout. If the next event arrives 20 seconds
> later, than you won't see the timeout until then.
>
> In the case of processing time, we could think about registering timeout
> timers for processing time. However, I would highly recommend you to use
> event time, because with processing time, Flink cannot guarantee meaningful
> computations, because the events might arrive out of order.
>
> Cheers,
> Till
>
> On Thu, Oct 6, 2016 at 3:08 PM, David Koch <ogdude@googlemail.com> wrote:
>
> Hello,
>
> With Flink CEP, is there a way to actively listen to pattern matches that
> time out? I am under the impression that this is not possible.
>
> In my case I partition a stream containing user web navigation by "userId"
> to look for sequences of Event A, followed by B within 4 seconds for each
> user.
>
> I registered a PatternTimeoutFunction which assuming a non-match only
> fires upon the first event after the specified timeout. For example, given
> user X: Event A, 20 seconds later Event B (or any other type of event).
>
> I'd rather have a notification fire directly upon the 4 second interval
> expiring since passive invalidation is not really applicable in my case.
>
> How, if at all can this be achieved with Flink CEP?
>
> Thanks,
>
> David
>
>
>
>
>
>
>

Mime
View raw message