giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Maja Kabiljo" <majakabi...@fb.com>
Subject Review Request: Fixing checkpointing for aggregators
Date Wed, 22 Aug 2012 13:05:19 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/6731/
-----------------------------------------------------------

Review request for giraph.


Description
-------

Making aggregators work correctly with checkpointing - saving the aggregator name, class,
value and whether it's persistent. Apart from that, I removed the code for aggregators handling
from BspServiceWorker and BspServiceMaster to separate classes, since I think it's cleaner
this way, and those two classes do too much different stuff as it is. But that's the reason
why the patch looks big. Later with GIRAPH-273 AggregatorHandler classes should become more
independent of BspServices.


This addresses bug GIRAPH-293.
    https://issues.apache.org/jira/browse/GIRAPH-293


Diffs
-----

  http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/bsp/CentralizedServiceMaster.java
1375970 
  http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/bsp/CentralizedServiceWorker.java
1375970 
  http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/graph/AggregatorHandler.java
PRE-CREATION 
  http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/graph/BspService.java
1375970 
  http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/graph/BspServiceMaster.java
1375970 
  http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/graph/BspServiceWorker.java
1375970 
  http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/graph/GraphMapper.java
1375970 
  http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/graph/MasterAggregatorHandler.java
PRE-CREATION 
  http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/graph/WorkerAggregatorHandler.java
PRE-CREATION 
  http://svn.apache.org/repos/asf/giraph/trunk/src/test/java/org/apache/giraph/TestAggregatorsHandling.java
1375970 
  http://svn.apache.org/repos/asf/giraph/trunk/src/test/java/org/apache/giraph/graph/TestAggregatorsHandling.java
PRE-CREATION 

Diff: https://reviews.apache.org/r/6731/diff/


Testing
-------

I added test for aggregator serialization and manual restarting from checkpoint (that one
also relies on recent GIRAPH-296 and GIRAPH-298 working). The patch passes mvn verify and
tests in pseudo-distributed mode.


Thanks,

Maja Kabiljo


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message