giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexander Frolov <alexndr.fro...@gmail.com>
Subject Re: Performance tuning of Giraph on small cluster with Infiniband
Date Fri, 07 Feb 2014 16:48:09 GMT
I forgot to note that in time scale it is only included time of computation
(e.i. sum of superstep times).

Yes, this is not a big graph, I will come up with larger graphs soon.

Best,
  Alex


On Fri, Feb 7, 2014 at 7:41 PM, Alexander Frolov
<alexndr.frolov@gmail.com>wrote:

> Undirected RMAT graph, generated by tool extracted from Graph500. Size is
> 2^20 vertices, average degree is 32.
>
>
> On Fri, Feb 7, 2014 at 5:35 PM, Claudio Martella <
> claudio.martella@gmail.com> wrote:
>
>> looks like a very small graph. what's the size of the graph and the
>> topology?
>>
>>
>> On Fri, Feb 7, 2014 at 3:11 PM, Alexander Frolov <
>> alexndr.frolov@gmail.com> wrote:
>>
>>> Hi, team!
>>>
>>> As I have read in previous threads, I've started evaluation of Giraph on
>>> IB-cluster. So here I want to share my results (in case it will be useful
>>> for anybody) and ask for your ideas of further improving of performance
>>> characteristics.
>>>
>>> Test system:
>>> * 8 Nodes, with dual Intel Xeon CPU E5-2630 (6 cores/CPU), 80GB
>>> * Infiniband FDR Dual-Port 4x
>>> * SUSE 11.2
>>> * jdk1.7.0_51
>>>
>>> At the moment I am performing experiment with
>>> SimpleShortestPathsComputation test on generated RMAT graph. I attach plot
>>> wich shows scalability of Giraph up to 32 workers.
>>>
>>> As can be seen from the plot up to 8 workers there is almost linear
>>> scalability and then (from 8 to 32) speed is not going up. For me it seems
>>> strange that using additional cores on nodes wont bring any performance
>>> gain to the execution time. Have anybody meet with such behaviour?
>>>
>>> Next I am going to use threads instead of workers for cores utilization.
>>>  Also I am going to switch to the Hadoop-RDMA project.
>>>
>>>
>>> If anybody has any suggestion how I can achieve maximum performance on
>>> Giraph on the cluster, I will be obliged to you ;-)
>>>
>>> Hope for your feedback.
>>>
>>> Best,
>>>   Alex
>>>
>>>
>>>
>>>
>>>
>>
>>
>> --
>>    Claudio Martella
>>
>>
>
>

Mime
View raw message