flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adrienne Kole <adrienneko...@gmail.com>
Subject AllWindowed vs Windowed with 1 key
Date Sun, 07 May 2017 21:23:45 GMT
Hi,

I am doing simple aggregation with a keyed and global windows in flink.
When I compare the keyed window aggregation with 1 key and global window
(which has parallelism 1) I would expect that both of them would have
similar performance.

However, keyed stream with 1 key performs with 2x more throughput than
global window.
My configuration is 8 node cluster, 16 core in each node, parallelism = 128.

AFAIK, Flink doesn't manage skew by default and uses hash function to
assign keys to partitions. So if I have 1 key only, it should go to only
one partition always, which is semantically similar to global windows in
flink.

What can be the reason behind this difference in performance?

Thanks,
Adrienne

Mime
View raw message