flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jone Zhang <joyoungzh...@gmail.com>
Subject Re: How can i merge more than one flink stream
Date Tue, 25 Jul 2017 06:35:07 GMT
Thanks for your reply. I have another question:
In my situation, each of the three streams contains a local timestamp
segment. How can I ensure that their timestamps are consistent in each time
window before the merging operation? And how to ensure the arrival of all
the streams with consistent timestamps in each time window?

Thanks.

2017-07-20 13:39 GMT+08:00 Kien Truong <duckientruong@gmail.com>:

> Hi,
>
> To expand on Fabian's answer, there's a few API for join.
>
> * connect - you have to provide a CoprocessFunction.
>
> * window join/cogroup - you provide  key selector functions, a time window
> and a join/cogroup function.
>
> With the first method, you have to write more code, in exchange for much
> more flexible join condition.
>
> Regards,
> Kien
>
> On Jul 20, 2017, at 01:55, Fabian Hueske <fhueske@gmail.com> wrote:
>>
>> Hi,
>>
>> there are basically two operations to merge streams.
>>
>> 1. Union simply merges the input streams such that the resulting stream
>> has the records of all input streams. Union is a built-in operator in the
>> DataStream API. For that all streams must have the same data type.
>> 2. Join connects records of streams according to a join condition. When
>> joining streams, this condition is often based on some time bounds. Join
>> usually needs to be manually implemented using a stateful
>> CoProcessFunction.
>>
>> Once the streams are unioned or joined, you can apply a time-window on
>> the result stream.
>>
>> Best, Fabian
>>
>> 2017-07-19 9:05 GMT+02:00 Jone Zhang <joyoungzhang@gmail.com>:
>>
>>> I have three data streams
>>> 1. app exposed and click
>>> 2. app download
>>> 3. app install
>>>
>>> How can i merge the streams to create a unified stream,then compute
>>> it on time-based windows
>>>
>>> Thanks
>>>
>>
>>

Mime
View raw message