flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ioannis Kontopoulos <kls.yan...@gmail.com>
Subject Cartesian product over windows
Date Thu, 16 Feb 2017 13:09:17 GMT
Hello everyone,

Given a stream of events (each event has a timestamp and a key), I want to
create all possible combinations of the keys in a window (sliding, event
time) and then process those combinations in parallel.

For example, if the stream contains events with keys 1,2,3,4 in a given
window and the possible combinations are:

1-2
1-3
1-4
2-3
2-4
3-4

and if the parallelism is set to 2, I want to have events with these keys:

1-2    2-3
1-3    2-4
1-4    3-4

You can see that there is some replication. So when I use the apply method
on a window it will have the keys separated like the example above.

Is there a way to do that?

Mime
View raw message