giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Claudio Martella <claudio.marte...@gmail.com>
Subject Re: Performance tuning of Giraph on small cluster with Infiniband
Date Fri, 07 Feb 2014 14:35:31 GMT
looks like a very small graph. what's the size of the graph and the
topology?


On Fri, Feb 7, 2014 at 3:11 PM, Alexander Frolov
<alexndr.frolov@gmail.com>wrote:

> Hi, team!
>
> As I have read in previous threads, I've started evaluation of Giraph on
> IB-cluster. So here I want to share my results (in case it will be useful
> for anybody) and ask for your ideas of further improving of performance
> characteristics.
>
> Test system:
> * 8 Nodes, with dual Intel Xeon CPU E5-2630 (6 cores/CPU), 80GB
> * Infiniband FDR Dual-Port 4x
> * SUSE 11.2
> * jdk1.7.0_51
>
> At the moment I am performing experiment with
> SimpleShortestPathsComputation test on generated RMAT graph. I attach plot
> wich shows scalability of Giraph up to 32 workers.
>
> As can be seen from the plot up to 8 workers there is almost linear
> scalability and then (from 8 to 32) speed is not going up. For me it seems
> strange that using additional cores on nodes wont bring any performance
> gain to the execution time. Have anybody meet with such behaviour?
>
> Next I am going to use threads instead of workers for cores utilization.
>  Also I am going to switch to the Hadoop-RDMA project.
>
>
> If anybody has any suggestion how I can achieve maximum performance on
> Giraph on the cluster, I will be obliged to you ;-)
>
> Hope for your feedback.
>
> Best,
>   Alex
>
>
>
>
>


-- 
   Claudio Martella

Mime
View raw message