giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Edunov" <edu...@gmail.com>
Subject Re: Review Request 23140: Fix checkpointing
Date Tue, 15 Jul 2014 21:08:59 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23140/
-----------------------------------------------------------

(Updated July 15, 2014, 9:08 p.m.)


Review request for giraph.


Changes
-------

Fixed CR issues


Repository: giraph-git


Description
-------

This fix merely makes checkpointing work again. 


Diffs (updated)
-----

  giraph-core/src/main/java/org/apache/giraph/aggregators/AggregatorWrapper.java 9613805 
  giraph-core/src/main/java/org/apache/giraph/bsp/BspService.java 2e35373 
  giraph-core/src/main/java/org/apache/giraph/comm/ServerData.java 85bfe04 
  giraph-core/src/main/java/org/apache/giraph/conf/GiraphConstants.java ab0570f 
  giraph-core/src/main/java/org/apache/giraph/master/BspServiceMaster.java 0275395 
  giraph-core/src/main/java/org/apache/giraph/partition/BasicPartitionOwner.java 545d1af 
  giraph-core/src/main/java/org/apache/giraph/partition/HashMasterPartitioner.java 240687e

  giraph-core/src/main/java/org/apache/giraph/partition/HashWorkerPartitioner.java d833895

  giraph-core/src/main/java/org/apache/giraph/partition/MasterGraphPartitioner.java 50c750a

  giraph-core/src/main/java/org/apache/giraph/partition/PartitionBalancer.java 3454d62 
  giraph-core/src/main/java/org/apache/giraph/partition/PartitionOwner.java 0ac74da 
  giraph-core/src/main/java/org/apache/giraph/partition/SimpleMasterPartitioner.java f128f34

  giraph-core/src/main/java/org/apache/giraph/partition/SimpleWorkerPartitioner.java 3c0de44

  giraph-core/src/main/java/org/apache/giraph/partition/WorkerGraphPartitioner.java 004ea81

  giraph-core/src/main/java/org/apache/giraph/utils/InternalVertexRunner.java 2c4606f 
  giraph-core/src/main/java/org/apache/giraph/utils/io/ExtendedDataInputOutput.java af45426

  giraph-core/src/main/java/org/apache/giraph/worker/BspServiceWorker.java de7af28 
  giraph-core/src/main/java/org/apache/giraph/worker/WorkerContext.java 29835c5 
  giraph-core/src/test/java/org/apache/giraph/partition/SimpleRangePartitionFactoryTest.java
96bd5d7 

Diff: https://reviews.apache.org/r/23140/diff/


Testing
-------

I tested it running multiple different jobs. I run page rank on 2*10^9 vertices on 200 workers
and it seems to work just fine. It only takes 2 minutes to save checkpoint. 


Thanks,

Sergey Edunov


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message