giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexander Frolov <alexndr.fro...@gmail.com>
Subject Re: Performance tuning of Giraph on small cluster with Infiniband
Date Fri, 07 Feb 2014 16:41:32 GMT
Undirected RMAT graph, generated by tool extracted from Graph500. Size is
2^20 vertices, average degree is 32.


On Fri, Feb 7, 2014 at 5:35 PM, Claudio Martella <claudio.martella@gmail.com
> wrote:

> looks like a very small graph. what's the size of the graph and the
> topology?
>
>
> On Fri, Feb 7, 2014 at 3:11 PM, Alexander Frolov <alexndr.frolov@gmail.com
> > wrote:
>
>> Hi, team!
>>
>> As I have read in previous threads, I've started evaluation of Giraph on
>> IB-cluster. So here I want to share my results (in case it will be useful
>> for anybody) and ask for your ideas of further improving of performance
>> characteristics.
>>
>> Test system:
>> * 8 Nodes, with dual Intel Xeon CPU E5-2630 (6 cores/CPU), 80GB
>> * Infiniband FDR Dual-Port 4x
>> * SUSE 11.2
>> * jdk1.7.0_51
>>
>> At the moment I am performing experiment with
>> SimpleShortestPathsComputation test on generated RMAT graph. I attach plot
>> wich shows scalability of Giraph up to 32 workers.
>>
>> As can be seen from the plot up to 8 workers there is almost linear
>> scalability and then (from 8 to 32) speed is not going up. For me it seems
>> strange that using additional cores on nodes wont bring any performance
>> gain to the execution time. Have anybody meet with such behaviour?
>>
>> Next I am going to use threads instead of workers for cores utilization.
>>  Also I am going to switch to the Hadoop-RDMA project.
>>
>>
>> If anybody has any suggestion how I can achieve maximum performance on
>> Giraph on the cluster, I will be obliged to you ;-)
>>
>> Hope for your feedback.
>>
>> Best,
>>   Alex
>>
>>
>>
>>
>>
>
>
> --
>    Claudio Martella
>
>

Mime
View raw message