giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexander Frolov <alexndr.fro...@gmail.com>
Subject Performance tuning of Giraph on small cluster with Infiniband
Date Fri, 07 Feb 2014 14:11:46 GMT
Hi, team!

As I have read in previous threads, I've started evaluation of Giraph on
IB-cluster. So here I want to share my results (in case it will be useful
for anybody) and ask for your ideas of further improving of performance
characteristics.

Test system:
* 8 Nodes, with dual Intel Xeon CPU E5-2630 (6 cores/CPU), 80GB
* Infiniband FDR Dual-Port 4x
* SUSE 11.2
* jdk1.7.0_51

At the moment I am performing experiment with
SimpleShortestPathsComputation test on generated RMAT graph. I attach plot
wich shows scalability of Giraph up to 32 workers.

As can be seen from the plot up to 8 workers there is almost linear
scalability and then (from 8 to 32) speed is not going up. For me it seems
strange that using additional cores on nodes wont bring any performance
gain to the execution time. Have anybody meet with such behaviour?

Next I am going to use threads instead of workers for cores utilization.
 Also I am going to switch to the Hadoop-RDMA project.


If anybody has any suggestion how I can achieve maximum performance on
Giraph on the cluster, I will be obliged to you ;-)

Hope for your feedback.

Best,
  Alex

Mime
View raw message