flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Till Rohrmann <trohrm...@apache.org>
Subject Re: Does Flink support joining multiple streams based on event time window now?
Date Tue, 19 Apr 2016 09:55:32 GMT
Hi Yifei,

if you don't wanna implement your own join operator, then you could also
chain two join operations. I created a small example to demonstrate that:
https://gist.github.com/tillrohrmann/c074b4eedb9deaf9c8ca2a5e124800f3.
However, bare in mind that for this approach you will construct two windows
which might be a bit more costly than Aljoscha's approach.

Cheers,
Till

On Tue, Apr 19, 2016 at 11:32 AM, Aljoscha Krettek <aljoscha@apache.org>
wrote:

> Hi,
> right now, there is no built-in support for n-ary joins. I am working on
> this, however.
>
> For now you can simulate n-ary joins by using a tagged union and doing the
> join yourself in a WindowFunction. I created a small example that
> demonstrates this:
> https://gist.github.com/aljoscha/a2a213d90c7c1bc67e71fabaa82fba4a
>
> I hope this helps, and please let us know if you want to know more.
>
> Cheers,
> Aljoscha
>
> On Tue, 19 Apr 2016 at 02:11 Yifei Li <lee891031@gmail.com> wrote:
>
>> Hi,
>>
>> I am new to Flink and I've read some documentation and think Flink may
>> fit my scenario.
>>
>> Here is my scenario:
>>
>> 1. Assume I have 3 streams: S1(id, name, email, action, date), S2(id,
>> name, email, level, date), S3(id, name, position, date).
>>
>> *2. S2 always delays(hours to days, not determined..) *
>>
>> 3. Based on the event time, I want to join S1, S2 and S3 every 5 minutes.
>> The join is like a SQL join:
>>     select S1.name, S3.position from S1, S2, S3 where S1.id = S2.id and
>> S1.id = S3.id and S1.action = 'download' and S2.level = 5
>>
>>
>>
>> Can I use Flink for my scenario? Is yes, can anyone point me to some
>> working examples(I found some examples but they are outdated), or tell me
>> some workaround to solve this problem? If no, can anyone tell me the
>> reasons?
>>
>> Thanks,
>>
>> Yifei
>>
>>
>>
>>
>>
>>

Mime
View raw message