giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Avery Ching" <avery.ch...@gmail.com>
Subject Re: Review Request: aggregator at input super step
Date Wed, 19 Jun 2013 00:27:06 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11525/#review22090
-----------------------------------------------------------


Minor issues below.  I was able to run 'mvn clean verify' with your code and it passed.  Let's
fix these minor things and ship it!


giraph-core/src/main/java/org/apache/giraph/master/BspServiceMaster.java
<https://reviews.apache.org/r/11525/#comment45478>

    cooperates with other code



giraph-core/src/main/java/org/apache/giraph/worker/EdgeInputSplitsCallable.java
<https://reviews.apache.org/r/11525/#comment45479>

    Aggregator handler



giraph-examples/src/main/java/org/apache/giraph/examples/GeneratedEdgeReader.java
<https://reviews.apache.org/r/11525/#comment45477>

    Can you please convert this to LongConfOption?  See GiraphConf for an example.


- Avery Ching


On June 17, 2013, 4:48 p.m., Bingjing Zhang wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/11525/
> -----------------------------------------------------------
> 
> (Updated June 17, 2013, 4:48 p.m.)
> 
> 
> Review request for giraph.
> 
> 
> Description
> -------
> 
> add aggregator for vertex reader and edge reader at input super step.
> 
> Changes in code:
> add "aggregate" interface to VertexReader and EdgeReader
> initialize "aggregator" in "readInputSplit" in "VertexInputSplitsCallable" and "EdgeInputSplitsCallable"
> initialize master aggregator before data loading in "coordinateSuperStep" in "BspServiceMaster"
> seperate master aggregator initialization and its computation for Super Step 0 (originally
they are mixed in one method "runMasterCompute")
> perpare aggregator in "setup" in "BspServiceWorker" before data loading
> 
> Test:
> Add SimpleVertexReader and  SimpleEdgeReader to AggregatorsTestComputation for genreating
input data.
> Also add "aggregate" call in "getCurrentVertex" and "getCurrentEdge".
> Register persistent aggregator with name "INPUT_VERTEX_PERSISTENT_AGG" and "INPUT_EDGE_PERSISTENT_AGG"in
AggregatorsTestMasterCompute.initialize
> Assert if aggregated values can be retrived in AggregatorsTestMasterCompute.compute when
"superstep >= 0"
> 
> 
> This addresses bug GIRAPH-673.
>     https://issues.apache.org/jira/browse/GIRAPH-673
> 
> 
> Diffs
> -----
> 
>   giraph-core/src/main/java/org/apache/giraph/io/EdgeReader.java 363a5e6 
>   giraph-core/src/main/java/org/apache/giraph/io/VertexReader.java b8b82af 
>   giraph-core/src/main/java/org/apache/giraph/io/internal/WrappedEdgeReader.java aae7a72

>   giraph-core/src/main/java/org/apache/giraph/io/internal/WrappedVertexReader.java 54adfec

>   giraph-core/src/main/java/org/apache/giraph/master/BspServiceMaster.java bd48116 
>   giraph-core/src/main/java/org/apache/giraph/worker/BspServiceWorker.java eb6d30d 
>   giraph-core/src/main/java/org/apache/giraph/worker/EdgeInputSplitsCallable.java 78cdd8e

>   giraph-core/src/main/java/org/apache/giraph/worker/VertexInputSplitsCallable.java 977e100

>   giraph-examples/src/main/java/org/apache/giraph/examples/AggregatorsTestComputation.java
db527f2 
>   giraph-examples/src/main/java/org/apache/giraph/examples/GeneratedEdgeReader.java PRE-CREATION

>   giraph-examples/src/test/java/org/apache/giraph/aggregators/TestAggregatorsHandling.java
6d22800 
> 
> Diff: https://reviews.apache.org/r/11525/diff/
> 
> 
> Testing
> -------
> 
> Unit Test of persistent aggregator at input super step in modified "AggregatorsTestComputation"
is done.
> "mvn clean verify" was done on giraph-core
> "mvn clean verfiy" was done on giraph-example
> 
> rechecked "indent" problem. Hopefully they are all solved
> 
> 
> Thanks,
> 
> Bingjing Zhang
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message