Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id C2D88200CE4 for ; Mon, 31 Jul 2017 04:24:45 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id C16AA163C4E; Mon, 31 Jul 2017 02:24:45 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id DEC84163C4A for ; Mon, 31 Jul 2017 04:24:44 +0200 (CEST) Received: (qmail 72340 invoked by uid 500); 31 Jul 2017 02:24:43 -0000 Mailing-List: contact user-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list user@flink.apache.org Received: (qmail 72328 invoked by uid 99); 31 Jul 2017 02:24:43 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 31 Jul 2017 02:24:43 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 4F0F1C006E for ; Mon, 31 Jul 2017 02:24:43 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 3.379 X-Spam-Level: *** X-Spam-Status: No, score=3.379 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_REPLY=1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RCVD_IN_SORBS_SPAM=0.5, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd4-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id fYJ54nIUUUCO for ; Mon, 31 Jul 2017 02:24:41 +0000 (UTC) Received: from mail-it0-f42.google.com (mail-it0-f42.google.com [209.85.214.42]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id D4E535F520 for ; Mon, 31 Jul 2017 02:24:40 +0000 (UTC) Received: by mail-it0-f42.google.com with SMTP id v127so119462010itd.0 for ; Sun, 30 Jul 2017 19:24:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=NS2AIulfymfKZyotYtWl/0dwjQcvyMrCnrD1KAoO9xg=; b=mCCHS6nl9t6ItsKa9WErPTmy697iDfqqdhalEihXOTLv3D6IKnuAM4nU0AMJ6jXM2y sYS6q/fl/tVeyLioG2e9EihzpoQ4Lnq/WEuBckvzc9M+2QahjtyMRKlgSupbih0T/6ke I1FI86M7LNem+eAI9i+NcQKfEIhAqmqOqaGnSMlfoCEejfFjzLx/hHGOR+sM3YyFZnIu +Msdrq2mUgUK8IXnIqR/C+ApuQbwuOjTCwpaG06UHkXBifNNp/kvN5gq50CHtAwBChgA wLkYvX96+xVdxWLRSVy5c7ahsujzkh56O6Ipvln5bzdhdUonT38HucEqcHnH6G2eOWT0 eswg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=NS2AIulfymfKZyotYtWl/0dwjQcvyMrCnrD1KAoO9xg=; b=VZWH/veLyIrp9BJwPTG+vvbKs/JbSnvN218JKX9fina8/5WAQ4lYvSifd+5PMWaLyf l5xh1D54DbIibs6JgYrecz+LF2vqc/3uqMsf/mbbn3n6UM+rzJUXsPpBNMFnepQv7CBK PHlTmID5iAT5LjhSj16R9kus8DfP60FeyYEzihBYhpuGWvYG5vB4C/Z4xrMAXdqMynEC REejfslHcMLOFIRmBEP2TZSOzZnIamgIYYUrXLlKzmIEjfHqdALY44gRmdN6Yw9FBFDt Y+rzqAceusLbhxJ7Ha8zw0O4uC79OLtqlBtnyC/HrRMiHut1Bh8HUrLhEBrx7uHUFaDr 8cXA== X-Gm-Message-State: AIVw111A/XuWw19UJnYpo3SoLySiEJntqMiCNL1jcs2fDYQq9M3sYwWf ekF09ZUUz+vTjie6gTeKEl4Hhtpf3g== X-Received: by 10.36.230.129 with SMTP id e123mr16807468ith.170.1501467874099; Sun, 30 Jul 2017 19:24:34 -0700 (PDT) MIME-Version: 1.0 Received: by 10.107.27.193 with HTTP; Sun, 30 Jul 2017 19:24:33 -0700 (PDT) Received: by 10.107.27.193 with HTTP; Sun, 30 Jul 2017 19:24:33 -0700 (PDT) In-Reply-To: References: <000601d3096c$32b902e0$982b08a0$@gmail.com> From: "G.S.Vijay Raajaa" Date: Mon, 31 Jul 2017 07:54:33 +0530 Message-ID: Subject: Re: AW: Is watermark used by joining two streams To: Elma Fangli Hsiung Cc: user , jixian01@googlemail.com Content-Type: multipart/alternative; boundary="94eb2c006f64470b4d055593bb07" archived-at: Mon, 31 Jul 2017 02:24:46 -0000 --94eb2c006f64470b4d055593bb07 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi Fabian, How do I order by the merge time. Let's say I merge the stream at T1. I wanted to drop T2 merge if T2 < T1. Now depending on the arrival of data from individual stream and the time at which the merge happens, they become out of order. Any thoughts will be really appreciated. Regards, Vijay Raajaa GS On Jul 31, 2017 1:14 AM, "wei" wrote: Hello Fabian, thank you for your answer! Does it mean that the operator will wait until get two watermarks from the input streams and emits then the =E2=80=9Cslower=E2=80=9D watermark? Best regards Wei *Von:* Fabian Hueske [mailto:fhueske@gmail.com] *Gesendet:* Sunday, July 30, 2017 11:17 AM *An:* xie wei *Cc:* user *Betreff:* Re: Is watermark used by joining two streams Periodic and punctuated watermarks only differ in the way that they are generated. Afterwards they are treated the same. An operator with two input streams will always sync its own watermarks to the watermarks of both input streams, i.e., to the "slower" watermark of both inputs. So if the left input says it is 12:14 and the right says it is 11:53, the operator will have a internal time of 11:53 and emit watermarks according to that time. Hope that helps, Fabian 2017-07-28 15:00 GMT+02:00 xie wei : Hello, i want to join two streams based on event time window, every stream has its own watermark, one has priodic watermark and the other has punctuated watermark. are the watermarks used to trigger the join? if yes, which one and how is it used? Thank you and best regards Wei --94eb2c006f64470b4d055593bb07 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi Fabian,

H= ow do I order by the merge time. Let's say I merge the stream at T1. I = wanted to drop T2 merge if T2 < T1. Now depending on the arrival of data= from individual stream and the time at which the merge happens, they becom= e out of order. Any thoughts will be really appreciated.

Regards,
Vijay Raaj= aa GS

= On Jul 31, 2017 1:14 AM, "wei" <jixian01@googlemail.com> wrote:
<= div class=3D"m_5414776999515404802WordSection1">

Hello Fabian,

=C2=A0

thank you for your= answer!

=C2=A0

Does it mean that the operator = will wait until get two watermarks from the input streams and emits then th= e =E2=80=9Cslower=E2=80=9D watermark?

=C2=A0

Be= st regards

Wei

=C2=A0

Von: Fabian Hueske [mailto:fhueske@gmail.com]
= Gesendet: Sunday, July 30, 2017 11:17 AM
An: xie wei
Cc: user
Betreff: Re: Is watermark used by joining two strea= ms

=C2=A0

P= eriodic and punctuated watermarks only differ in the way that they are gene= rated. Afterwards they are treated the same.

An operator with two input streams will always sync its own= watermarks to the watermarks of both input streams, i.e., to the "slo= wer" watermark of both inputs.

So if the left input says it is 12:14= and the right says it is 11:53, the operator will have a internal time of = 11:53 and emit watermarks according to that time.

Hope that helps,

Fabian

=C2=A0

=C2=A0

2017= -07-28 15:00 GMT+02:00 xie wei <jixian01@googlemail.com>:

= Hello,

i want to join two str= eams based on event time window, every stream has its own watermark, one ha= s priodic watermark and the other has punctuated watermark.

are the water= marks used to trigger the join? if yes, which one and how is it used?

Thank you and best regards<= u>

Wei

=C2=A0

=C2=A0

<= /div>

--94eb2c006f64470b4d055593bb07--