giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bingjing Zhang" <jesse...@gmail.com>
Subject Re: Review Request: aggregator at input super step
Date Wed, 29 May 2013 17:41:35 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11525/
-----------------------------------------------------------

(Updated May 29, 2013, 5:41 p.m.)


Review request for giraph.


Description (updated)
-------

add aggregator for vertex reader and edge reader at input super step.

Changes in code:
add "aggregate" interface to VertexReader and EdgeReader
initialize "aggregator" in "readInputSplit" in "VertexInputSplitsCallable" and "EdgeInputSplitsCallable"
initialize master aggregator before data loading in "coordinateSuperStep" in "BspServiceMaster"
seperate master aggregator initialization and its computation for Super Step 0 (originally
they are mixed in one method "runMasterCompute")
perpare aggregator in "setup" in "BspServiceWorker" before data loading

Test:
Add SimpleVertexReader and  SimpleEdgeReader to AggregatorsTestComputation for genreating
input data.
Also add "aggregate" call in "getCurrentVertex" and "getCurrentEdge".
Register persistent aggregator with name "INPUT_VERTEX_PERSISTENT_AGG" and "INPUT_EDGE_PERSISTENT_AGG"in
AggregatorsTestMasterCompute.initialize
Assert if aggregated values can be retrived in AggregatorsTestMasterCompute.compute when "superstep
>= 0"


This addresses bug GIRAPH-673.
    https://issues.apache.org/jira/browse/GIRAPH-673


Diffs
-----

  giraph-core/src/main/java/org/apache/giraph/io/EdgeReader.java 363a5e6 
  giraph-core/src/main/java/org/apache/giraph/io/VertexReader.java 9695169 
  giraph-core/src/main/java/org/apache/giraph/io/internal/WrappedEdgeReader.java aae7a72 
  giraph-core/src/main/java/org/apache/giraph/io/internal/WrappedVertexReader.java 54adfec

  giraph-core/src/main/java/org/apache/giraph/master/BspServiceMaster.java bd48116 
  giraph-core/src/main/java/org/apache/giraph/worker/BspServiceWorker.java 8b5e39a 
  giraph-core/src/main/java/org/apache/giraph/worker/EdgeInputSplitsCallable.java 78cdd8e

  giraph-core/src/main/java/org/apache/giraph/worker/VertexInputSplitsCallable.java 977e100

  giraph-core/src/test/java/org/apache/giraph/master/TestSwitchClasses.java 4069972 
  giraph-examples/src/main/java/org/apache/giraph/examples/AggregatorsTestComputation.java
db527f2 
  giraph-examples/src/test/java/org/apache/giraph/aggregators/TestAggregatorsHandling.java
6d22800 

Diff: https://reviews.apache.org/r/11525/diff/


Testing (updated)
-------

Unit Test of persistent aggregator at input super step in modified "AggregatorsTestComputation"
is done.


Thanks,

Bingjing Zhang


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message