Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id C13FF200B63 for ; Mon, 15 Aug 2016 09:48:36 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id BFBB8160AA7; Mon, 15 Aug 2016 07:48:36 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 1133D160A6C for ; Mon, 15 Aug 2016 09:48:35 +0200 (CEST) Received: (qmail 88551 invoked by uid 500); 15 Aug 2016 07:48:35 -0000 Mailing-List: contact user-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@flink.apache.org Delivered-To: mailing list user@flink.apache.org Received: (qmail 88542 invoked by uid 99); 15 Aug 2016 07:48:35 -0000 Received: from mail-relay.apache.org (HELO mail-relay.apache.org) (140.211.11.15) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 15 Aug 2016 07:48:35 +0000 Received: from mail-wm0-f43.google.com (mail-wm0-f43.google.com [74.125.82.43]) by mail-relay.apache.org (ASF Mail Server at mail-relay.apache.org) with ESMTPSA id A72311A0055 for ; Mon, 15 Aug 2016 07:48:34 +0000 (UTC) Received: by mail-wm0-f43.google.com with SMTP id q128so76633606wma.1 for ; Mon, 15 Aug 2016 00:48:34 -0700 (PDT) X-Gm-Message-State: AEkoouvMvmKB2j/PKJp0vLIwXRt4cz9syAbPvgAl/FzTHt/bAq3+QpFRDTrha5pIlaKNk6VH9bVUXnpW8u2dCA== X-Received: by 10.28.72.139 with SMTP id v133mr13886824wma.32.1471247313074; Mon, 15 Aug 2016 00:48:33 -0700 (PDT) MIME-Version: 1.0 Received: by 10.194.81.233 with HTTP; Mon, 15 Aug 2016 00:48:32 -0700 (PDT) In-Reply-To: References: From: Till Rohrmann Date: Mon, 15 Aug 2016 09:48:32 +0200 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Within interval for CEP - Wall Clock based or Event Timestamp based? To: user@flink.apache.org Content-Type: multipart/alternative; boundary=001a114b32d87900f3053a177589 archived-at: Mon, 15 Aug 2016 07:48:36 -0000 --001a114b32d87900f3053a177589 Content-Type: text/plain; charset=UTF-8 Hi, at the moment the CEP operator using EventTime only processes elements after it has received a watermark which is larger than the timestamp of the elements. In case of late arrivals it might make sense to directly process these events. This should decrease the latency a bit (for this case). Cheers, Till On Fri, Aug 12, 2016 at 10:29 PM, Sameer W wrote: > Thanks Max - Especially the last part about late events. > > Sameer > > On Fri, Aug 12, 2016 at 4:23 AM, Maximilian Michels > wrote: > >> Hi Sameer, >> >> That depends on the time characteristic you have chosen. If you have >> set it to event time [1] then it will use event time, otherwise the >> default is to use processing time. >> >> When using event time, the element's timestamp is used to assign it to >> the specified time windows in the patterns, regardless of the time it >> actually arrives. The only exception being if the element arrives >> after the Watermark, then only the single element will be processed >> according to the patterns. >> >> Cheers, >> Max >> >> [1] env.setStreamTimeCharacteristic(TimeCharacteristic.EventTime); >> >> On Thu, Aug 11, 2016 at 12:39 AM, Sameer W wrote: >> > Hi, >> > >> > I am using EventTime but when the records get into the CEP PatternStream >> > does the WITHIN interval refer to the wall clock time or the timestamps >> > embedded in the event stream? >> > >> > If I provide WITHIN(Time.Seconds(5)) and in processing time I am getting >> > events with timestamps in the range of 10 seconds (due to upstream >> emits), >> > are all those events considered part of the pattern of their timestamps >> > matter? >> > >> > Thanks, >> > Sameer >> > > --001a114b32d87900f3053a177589 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Hi,

at the moment the CEP operator usin= g EventTime only processes elements after it has received a watermark which= is larger than the timestamp of the elements. In case of late arrivals it = might make sense to directly process these events. This should decrease the= latency a bit (for this case).

Cheers,
= Till

O= n Fri, Aug 12, 2016 at 10:29 PM, Sameer W <sameer@axiomine.com> wrote:
Thanks Max = - Especially the last part about late events.=C2=A0<= font color=3D"#888888">

Sameer
=

On Fri, Aug 12, 2016 at 4:23 AM, Maximilian Michels = <m= xm@apache.org> wrote:
Hi Sa= meer,

That depends on the time characteristic you have chosen. If you have
set it to event time [1] then it will use event time, otherwise the
default is to use processing time.

When using event time, the element's timestamp is used to assign it to<= br> the specified time windows in the patterns, regardless of the time it
actually arrives. The only exception being if the element arrives
after the Watermark, then only the single element will be processed
according to the patterns.

Cheers,
Max

[1] env.setStreamTimeCharacteristic(TimeCharacteristic.EventTime)= ;

On Thu, Aug 11, 2016 at 12:39 AM, Sameer W <sameer@axiomine.com> wrote:
> Hi,
>
> I am using EventTime but when the records get into the CEP PatternStre= am
> does the WITHIN interval refer to the wall clock time or the timestamp= s
> embedded in the event stream?
>
> If I provide WITHIN(Time.Seconds(5)) and in processing time I am getti= ng
> events with timestamps in the range of 10 seconds (due to upstream emi= ts),
> are all those events considered part of the pattern of their timestamp= s
> matter?
>
> Thanks,
> Sameer


--001a114b32d87900f3053a177589--