giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sebastian Schelter <...@apache.org>
Subject Re: Performance tuning of Giraph on small cluster with Infiniband
Date Fri, 07 Feb 2014 14:25:21 GMT
Hi Alexander,

how large is the graph that you run your test on in terms of vertices 
and edges?

--sebastian

On 02/07/2014 03:11 PM, Alexander Frolov wrote:
> Hi, team!
>
> As I have read in previous threads, I've started evaluation of Giraph on
> IB-cluster. So here I want to share my results (in case it will be useful
> for anybody) and ask for your ideas of further improving of performance
> characteristics.
>
> Test system:
> * 8 Nodes, with dual Intel Xeon CPU E5-2630 (6 cores/CPU), 80GB
> * Infiniband FDR Dual-Port 4x
> * SUSE 11.2
> * jdk1.7.0_51
>
> At the moment I am performing experiment with
> SimpleShortestPathsComputation test on generated RMAT graph. I attach plot
> wich shows scalability of Giraph up to 32 workers.
>
> As can be seen from the plot up to 8 workers there is almost linear
> scalability and then (from 8 to 32) speed is not going up. For me it seems
> strange that using additional cores on nodes wont bring any performance
> gain to the execution time. Have anybody meet with such behaviour?
>
> Next I am going to use threads instead of workers for cores utilization.
>   Also I am going to switch to the Hadoop-RDMA project.
>
>
> If anybody has any suggestion how I can achieve maximum performance on
> Giraph on the cluster, I will be obliged to you ;-)
>
> Hope for your feedback.
>
> Best,
>    Alex
>


Mime
View raw message