giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Avery Ching" <avery.ch...@gmail.com>
Subject Re: Review Request: More SendMessageCache improvements
Date Mon, 05 Nov 2012 23:20:04 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7883/#review13115
-----------------------------------------------------------

Ship it!


+1!  This is pretty clean.  


http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/main/java/org/apache/giraph/comm/SendMessageCache.java
<https://reviews.apache.org/r/7883/#comment28254>

    This comment should be fixed.



http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/main/java/org/apache/giraph/comm/VertexIdMessageCollection.java
<https://reviews.apache.org/r/7883/#comment28263>

    Nice refactoring.


- Avery Ching


On Nov. 5, 2012, 10:59 p.m., Maja Kabiljo wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/7883/
> -----------------------------------------------------------
> 
> (Updated Nov. 5, 2012, 10:59 p.m.)
> 
> 
> Review request for giraph.
> 
> 
> Description
> -------
> 
> Having a lot of maps in SendMessageCache still makes it slow, so here is another step
towards making it faster.
> 
> Here are the results on PageRankBenchmark:
> 10m vertices, 100 edges per vertex, 10 workers
> 1 thread: Total superstep time: 54s -> 35s
> 20m vertices, 100 edges per vertex, 12 workers
> 4 threads: Computation time: 26s -> 17s
> 
> Also tested on one of our real applications, speedup was a bit smaller, about 20-25%.
> 
> 
> This addresses bug GIRAPH-404.
>     https://issues.apache.org/jira/browse/GIRAPH-404
> 
> 
> Diffs
> -----
> 
>   http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/main/java/org/apache/giraph/bsp/CentralizedServiceWorker.java
1405925 
>   http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/main/java/org/apache/giraph/comm/SendMessageCache.java
1405925 
>   http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/main/java/org/apache/giraph/comm/VertexIdMessageCollection.java
1405925 
>   http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/main/java/org/apache/giraph/comm/messages/DiskBackedMessageStoreByPartition.java
1405925 
>   http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/main/java/org/apache/giraph/comm/messages/SimpleMessageStore.java
1405925 
>   http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/main/java/org/apache/giraph/comm/netty/NettyWorkerClientRequestProcessor.java
1405925 
>   http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/main/java/org/apache/giraph/comm/requests/SendWorkerMessagesRequest.java
1405925 
>   http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/main/java/org/apache/giraph/graph/BspServiceWorker.java
1405925 
>   http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/main/java/org/apache/giraph/utils/PairList.java
PRE-CREATION 
>   http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/main/java/org/apache/giraph/utils/PairListWritable.java
PRE-CREATION 
>   http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/test/java/org/apache/giraph/comm/RequestFailureTest.java
1405925 
>   http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/test/java/org/apache/giraph/comm/RequestTest.java
1405925 
> 
> Diff: https://reviews.apache.org/r/7883/diff/
> 
> 
> Testing
> -------
> 
> mvn clean verify, pseudo-distributed tests
> PageRankBenchmark (results above)
> Tried it out with a real application
> 
> 
> Thanks,
> 
> Maja Kabiljo
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message