flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kostas Kloudas (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-6069) Add documentation CEP - Watermarks
Date Tue, 28 Mar 2017 09:26:41 GMT

    [ https://issues.apache.org/jira/browse/FLINK-6069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15944856#comment-15944856

Kostas Kloudas commented on FLINK-6069:

Hi [~patricklx] ,

Currently the CEP library had a somehow fuzzy way to handle late events. Essentially:
1) it was accepting all events (late and early)
2) whenever a watermark was arriving it was sorting them based on event time
3) it was feeding them into the NFA.

As you see this is not respecting event time. 

I will open a PR  soon (already working on it) where the underlying assumption is that the
watermark is correct. This means that late events are discarded. Late events are the ones
whose timestamp is smaller than the last received watermark. 

Dropped events will be sent to a side output, as done in the case of the {{WindowOperator}}.

> Add documentation CEP - Watermarks
> ----------------------------------
>                 Key: FLINK-6069
>                 URL: https://issues.apache.org/jira/browse/FLINK-6069
>             Project: Flink
>          Issue Type: Improvement
>          Components: CEP
>    Affects Versions: 1.2.0, 1.3.0
>            Reporter: Patrick Pircher
>            Assignee: Kostas Kloudas
>            Priority: Minor
> I was trying to understand how CEP works with EventTime/Watermarks and specifically how
it handles events that come out of order.
> So the only thing I found was an email archive and some pointers in the Source Code.
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/CEP-and-slightly-out-of-order-elements-td9439.html
> https://github.com/apache/flink/blob/release-1.2.0/flink-libraries/flink-cep/src/main/java/org/apache/flink/cep/operator/AbstractCEPPatternOperator.java#L71
> https://github.com/apache/flink/blob/release-1.2.0/flink-libraries/flink-cep/src/main/java/org/apache/flink/cep/operator/AbstractCEPPatternOperator.java#L96

This message was sent by Atlassian JIRA

View raw message