flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kaepke, Marc" <marc.kae...@haw-hamburg.de>
Subject Re: PageRank - 4x slower then Spark?!
Date Wed, 23 Aug 2017 15:11:21 GMT
Does someone has a current performance test based on PageRank or an idea why Flink lost the
comparison?


> Am 18.08.2017 um 19:51 schrieb Kaepke, Marc <marc.kaepke@haw-hamburg.de>:
> 
> Hi everyone,
> 
> I compared Flink and Spark by using PageRank. I guessed Flink will beat Spark or have
the same level. But Spark is up to 4x faster then Flink.
> I hope I did a mistake. So please help me to improve the performance of my cluster and
config.
> 
> The cluster has 4 computers:
> One JobManager (Quad Core with Hyper Threading -> 8 cores) and 16GB jobmanager.heap.mp))
> Three TaskManager (each Quad Core with Hyper Threading -> 8 cores and 16GB (taskmanager.heap.mp))
> In total 24 cores/ task slots.
> 
> I ran PR as vertex-centric, scatter-gather, gather-sum-apply and with bulk iteration.
The parallelism was 24.
> Runtime in ms:
> Pregel: 90.000ms
> SG: 64.000ms
> GSA: 80.000ms
> Bulk: 53.000ms
> Spark with Pregel ran in 23.000ms
> 
> The input file was: https://snap.stanford.edu/data/wiki-topcats.html
> 
> Thanks for helping!
> 
> Marc


Mime
View raw message