flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "G.S.Vijay Raajaa" <gsvijayraa...@gmail.com>
Subject Re: AW: Is watermark used by joining two streams
Date Mon, 31 Jul 2017 02:24:33 GMT
Hi Fabian,

How do I order by the merge time. Let's say I merge the stream at T1. I
wanted to drop T2 merge if T2 < T1. Now depending on the arrival of data
from individual stream and the time at which the merge happens, they become
out of order. Any thoughts will be really appreciated.

Regards,
Vijay Raajaa GS

On Jul 31, 2017 1:14 AM, "wei" <jixian01@googlemail.com> wrote:

Hello Fabian,



thank you for your answer!



Does it mean that the operator will wait until get two watermarks from the
input streams and emits then the “slower” watermark?



Best regards

Wei



*Von:* Fabian Hueske [mailto:fhueske@gmail.com]
*Gesendet:* Sunday, July 30, 2017 11:17 AM
*An:* xie wei
*Cc:* user
*Betreff:* Re: Is watermark used by joining two streams



Periodic and punctuated watermarks only differ in the way that they are
generated. Afterwards they are treated the same.

An operator with two input streams will always sync its own watermarks to
the watermarks of both input streams, i.e., to the "slower" watermark of
both inputs.

So if the left input says it is 12:14 and the right says it is 11:53, the
operator will have a internal time of 11:53 and emit watermarks according
to that time.

Hope that helps,

Fabian





2017-07-28 15:00 GMT+02:00 xie wei <jixian01@googlemail.com>:

Hello,

i want to join two streams based on event time window, every stream has its
own watermark, one has priodic watermark and the other has punctuated
watermark.

are the watermarks used to trigger the join? if yes, which one and how is
it used?

Thank you and best regards

Wei

Mime
View raw message