giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Avery Ching" <avery.ch...@gmail.com>
Subject Re: Review Request: aggregator at input super step
Date Thu, 13 Jun 2013 15:13:00 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11525/#review21844
-----------------------------------------------------------


Getting closer! =)

I tried to run mvn clean verify with your patch, but ran into the following build issues:

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.0:compile
(default-compile) on project giraph-examples: Compilation failure: Compilation failure:
[ERROR] /Users/aching/git/giraph_git/giraph-examples/target/munged/main/org/apache/giraph/examples/AggregatorsTestComputation.java:[231,5]
cannot find symbol
[ERROR] symbol  : class GeneratedEdgeReader
[ERROR] location: class org.apache.giraph.examples.AggregatorsTestComputation
[ERROR] /Users/aching/git/giraph_git/giraph-examples/target/munged/main/org/apache/giraph/examples/AggregatorsTestComputation.java:[237,14]
cannot find symbol
[ERROR] symbol  : variable totalRecords
[ERROR] location: class org.apache.giraph.examples.AggregatorsTestComputation.SimpleEdgeReader
[ERROR] /Users/aching/git/giraph_git/giraph-examples/target/munged/main/org/apache/giraph/examples/AggregatorsTestComputation.java:[237,29]
cannot find symbol
[ERROR] symbol  : variable recordsRead
[ERROR] location: class org.apache.giraph.examples.AggregatorsTestComputation.SimpleEdgeReader

I think you are missing GeneratedEdgeReader in this patch?


giraph-core/src/main/java/org/apache/giraph/master/BspServiceMaster.java
<https://reviews.apache.org/r/11525/#comment45130>

    Called 2x?



giraph-core/src/main/java/org/apache/giraph/master/BspServiceMaster.java
<https://reviews.apache.org/r/11525/#comment45129>

    is  called -> is called



giraph-core/src/main/java/org/apache/giraph/io/VertexReader.java
<https://reviews.apache.org/r/11525/#comment45132>

    This extra line is not needed, see other classes.



giraph-core/src/main/java/org/apache/giraph/master/BspServiceMaster.java
<https://reviews.apache.org/r/11525/#comment45133>

    INPUT Super Step -> input superstep or INPUT_SUPERSTEP, your choice =)



giraph-core/src/main/java/org/apache/giraph/master/BspServiceMaster.java
<https://reviews.apache.org/r/11525/#comment45134>

    can we make this initializeAggregatorInputSuperstep?  In other parts of the code, superstep
is a single word.



giraph-core/src/main/java/org/apache/giraph/master/BspServiceMaster.java
<https://reviews.apache.org/r/11525/#comment45135>

    initializeMasterCompute -> initializeAggregatorInputSuperstep



giraph-core/src/main/java/org/apache/giraph/master/BspServiceMaster.java
<https://reviews.apache.org/r/11525/#comment45136>

    initializeMasterCompute -> initializeAggregatorInputSuperstep



giraph-core/src/main/java/org/apache/giraph/master/BspServiceMaster.java
<https://reviews.apache.org/r/11525/#comment45137>

    initializeMasterCompute -> initializeAggregatorInputSuperstep



giraph-core/src/main/java/org/apache/giraph/master/BspServiceMaster.java
<https://reviews.apache.org/r/11525/#comment45138>

    runMasterCompute -> initializeAggregatorInputSuperstep



giraph-core/src/main/java/org/apache/giraph/master/BspServiceMaster.java
<https://reviews.apache.org/r/11525/#comment45139>

    Please remove this line, it only describes what you did in this diff.


- Avery Ching


On June 12, 2013, 3:25 a.m., Bingjing Zhang wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/11525/
> -----------------------------------------------------------
> 
> (Updated June 12, 2013, 3:25 a.m.)
> 
> 
> Review request for giraph.
> 
> 
> Description
> -------
> 
> add aggregator for vertex reader and edge reader at input super step.
> 
> Changes in code:
> add "aggregate" interface to VertexReader and EdgeReader
> initialize "aggregator" in "readInputSplit" in "VertexInputSplitsCallable" and "EdgeInputSplitsCallable"
> initialize master aggregator before data loading in "coordinateSuperStep" in "BspServiceMaster"
> seperate master aggregator initialization and its computation for Super Step 0 (originally
they are mixed in one method "runMasterCompute")
> perpare aggregator in "setup" in "BspServiceWorker" before data loading
> 
> Test:
> Add SimpleVertexReader and  SimpleEdgeReader to AggregatorsTestComputation for genreating
input data.
> Also add "aggregate" call in "getCurrentVertex" and "getCurrentEdge".
> Register persistent aggregator with name "INPUT_VERTEX_PERSISTENT_AGG" and "INPUT_EDGE_PERSISTENT_AGG"in
AggregatorsTestMasterCompute.initialize
> Assert if aggregated values can be retrived in AggregatorsTestMasterCompute.compute when
"superstep >= 0"
> 
> 
> This addresses bug GIRAPH-673.
>     https://issues.apache.org/jira/browse/GIRAPH-673
> 
> 
> Diffs
> -----
> 
>   giraph-core/src/main/java/org/apache/giraph/io/EdgeReader.java 363a5e6 
>   giraph-core/src/main/java/org/apache/giraph/io/VertexReader.java 9695169 
>   giraph-core/src/main/java/org/apache/giraph/io/internal/WrappedEdgeReader.java aae7a72

>   giraph-core/src/main/java/org/apache/giraph/io/internal/WrappedVertexReader.java 54adfec

>   giraph-core/src/main/java/org/apache/giraph/master/BspServiceMaster.java bd48116 
>   giraph-core/src/main/java/org/apache/giraph/worker/BspServiceWorker.java 8b5e39a 
>   giraph-core/src/main/java/org/apache/giraph/worker/EdgeInputSplitsCallable.java 78cdd8e

>   giraph-core/src/main/java/org/apache/giraph/worker/VertexInputSplitsCallable.java 977e100

>   giraph-examples/src/main/java/org/apache/giraph/examples/AggregatorsTestComputation.java
db527f2 
>   giraph-examples/src/test/java/org/apache/giraph/aggregators/TestAggregatorsHandling.java
6d22800 
> 
> Diff: https://reviews.apache.org/r/11525/diff/
> 
> 
> Testing
> -------
> 
> Unit Test of persistent aggregator at input super step in modified "AggregatorsTestComputation"
is done.
> "mvn clean verify" was done on giraph-core
> "mvn clean verfiy" was done on giraph-example
> 
> rechecked "indent" problem. Hopefully they are all solved
> 
> 
> Thanks,
> 
> Bingjing Zhang
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message