Return-Path: X-Original-To: apmail-flink-commits-archive@minotaur.apache.org Delivered-To: apmail-flink-commits-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 8380C18BE3 for ; Fri, 26 Feb 2016 12:33:11 +0000 (UTC) Received: (qmail 65322 invoked by uid 500); 26 Feb 2016 12:33:01 -0000 Delivered-To: apmail-flink-commits-archive@flink.apache.org Received: (qmail 65283 invoked by uid 500); 26 Feb 2016 12:33:01 -0000 Mailing-List: contact commits-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@flink.apache.org Delivered-To: mailing list commits@flink.apache.org Received: (qmail 65274 invoked by uid 99); 26 Feb 2016 12:33:01 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 26 Feb 2016 12:33:01 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 8724ADFCE0; Fri, 26 Feb 2016 12:33:01 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: sewen@apache.org To: commits@flink.apache.org Message-Id: <97a9febb9a154d47b52f9329ee72a6bb@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: flink git commit: [FLINK-3520] [streaming] Periodic watermark operator emits current watermark in close() Date: Fri, 26 Feb 2016 12:33:01 +0000 (UTC) Repository: flink Updated Branches: refs/heads/master 725f7467e -> de9a878e9 [FLINK-3520] [streaming] Periodic watermark operator emits current watermark in close() This makes sure that for bounded data sets with watermarks, the final elements get properly reflected in window results. Project: http://git-wip-us.apache.org/repos/asf/flink/repo Commit: http://git-wip-us.apache.org/repos/asf/flink/commit/de9a878e Tree: http://git-wip-us.apache.org/repos/asf/flink/tree/de9a878e Diff: http://git-wip-us.apache.org/repos/asf/flink/diff/de9a878e Branch: refs/heads/master Commit: de9a878e9555023e1c63f9c41eb281650f4840df Parents: 725f746 Author: Stephan Ewen Authored: Fri Feb 26 12:01:44 2016 +0100 Committer: Stephan Ewen Committed: Fri Feb 26 13:32:27 2016 +0100 ---------------------------------------------------------------------- .../TimestampsAndPeriodicWatermarksOperator.java | 11 +++++++++++ 1 file changed, 11 insertions(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/flink/blob/de9a878e/flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/operators/TimestampsAndPeriodicWatermarksOperator.java ---------------------------------------------------------------------- diff --git a/flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/operators/TimestampsAndPeriodicWatermarksOperator.java b/flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/operators/TimestampsAndPeriodicWatermarksOperator.java index fbd46ef..a1614c0 100644 --- a/flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/operators/TimestampsAndPeriodicWatermarksOperator.java +++ b/flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/operators/TimestampsAndPeriodicWatermarksOperator.java @@ -88,4 +88,15 @@ public class TimestampsAndPeriodicWatermarksOperator output.emitWatermark(mark); } } + + @Override + public void close() throws Exception { + // emit a final watermark + Watermark newWatermark = userFunction.getCurrentWatermark(); + if (newWatermark != null && newWatermark.getTimestamp() > currentWatermark) { + currentWatermark = newWatermark.getTimestamp(); + // emit watermark + output.emitWatermark(newWatermark); + } + } }