giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alessandro Presta" <alessan...@fb.com>
Subject Re: Review Request: Allow creation of graph by adding edges that span multiple workers
Date Fri, 02 Nov 2012 21:30:59 GMT


> On Nov. 2, 2012, 9:12 p.m., Avery Ching wrote:
> > /trunk/giraph/src/main/java/org/apache/giraph/graph/BspServiceMaster.java, line
255
> > <https://reviews.apache.org/r/7784/diff/3/?file=184281#file184281line255>
> >
> >     prefer this in the exception to avoid runtime overhead without exceptions.
> >     
> >
> 
> Alessandro Presta wrote:
>     Not sure I follow. This is used also outside of the catch block.
> 
> Avery Ching wrote:
>     Okay, I guess we can live with this.  It just looks a little ugly, but it's okay.

Agreed, a few of the solutions I came up with for code sharing are not great, but better than
duplicating all that code.


> On Nov. 2, 2012, 9:12 p.m., Avery Ching wrote:
> > /trunk/giraph/src/main/java/org/apache/giraph/graph/BspUtils.java, lines 188-222
> > <https://reviews.apache.org/r/7784/diff/3/?file=184283#file184283line188>
> >
> >     why do you need these?  You already implemented this in ImmutableClassesGiraphConfiguration.
> 
> Alessandro Presta wrote:
>     You're right, it was just for consistency.
>     Should I go ahead and removed all the unused methods from BspUtils then?
> 
> Avery Ching wrote:
>     Maybe it's time we just removed BspUtils?  We can do it in another diff.

Sounds good


- Alessandro


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7784/#review13061
-----------------------------------------------------------


On Nov. 2, 2012, 9:28 p.m., Alessandro Presta wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/7784/
> -----------------------------------------------------------
> 
> (Updated Nov. 2, 2012, 9:28 p.m.)
> 
> 
> Review request for giraph.
> 
> 
> Description
> -------
> 
> Allow creation of graph by adding edges that span multiple workers
> 
> 
> This addresses bug GIRAPH-155.
>     https://issues.apache.org/jira/browse/GIRAPH-155
> 
> 
> Diffs
> -----
> 
>   /trunk/giraph/src/main/java/org/apache/giraph/GiraphConfiguration.java 1404751 
>   /trunk/giraph/src/main/java/org/apache/giraph/ImmutableClassesGiraphConfiguration.java
1404751 
>   /trunk/giraph/src/main/java/org/apache/giraph/bsp/CentralizedServiceMaster.java 1404751

>   /trunk/giraph/src/main/java/org/apache/giraph/comm/WorkerServer.java 1404751 
>   /trunk/giraph/src/main/java/org/apache/giraph/comm/netty/NettyWorkerClientRequestProcessor.java
1404751 
>   /trunk/giraph/src/main/java/org/apache/giraph/comm/netty/NettyWorkerServer.java 1404751

>   /trunk/giraph/src/main/java/org/apache/giraph/comm/requests/SendPartitionMutationsRequest.java
1404751 
>   /trunk/giraph/src/main/java/org/apache/giraph/graph/BasicVertexValueReader.java PRE-CREATION

>   /trunk/giraph/src/main/java/org/apache/giraph/graph/BspService.java 1404751 
>   /trunk/giraph/src/main/java/org/apache/giraph/graph/BspServiceMaster.java 1404751 
>   /trunk/giraph/src/main/java/org/apache/giraph/graph/BspServiceWorker.java 1404751 
>   /trunk/giraph/src/main/java/org/apache/giraph/graph/BspUtils.java 1404751 
>   /trunk/giraph/src/main/java/org/apache/giraph/graph/EdgeInputFormat.java PRE-CREATION

>   /trunk/giraph/src/main/java/org/apache/giraph/graph/EdgeInputSplitsCallable.java PRE-CREATION

>   /trunk/giraph/src/main/java/org/apache/giraph/graph/EdgeInputSplitsCallableFactory.java
PRE-CREATION 
>   /trunk/giraph/src/main/java/org/apache/giraph/graph/EdgeReader.java PRE-CREATION 
>   /trunk/giraph/src/main/java/org/apache/giraph/graph/EdgeWithSource.java PRE-CREATION

>   /trunk/giraph/src/main/java/org/apache/giraph/graph/GiraphInputFormat.java PRE-CREATION

>   /trunk/giraph/src/main/java/org/apache/giraph/graph/GiraphJob.java 1404751 
>   /trunk/giraph/src/main/java/org/apache/giraph/graph/GraphMapper.java 1404751 
>   /trunk/giraph/src/main/java/org/apache/giraph/graph/InputSplitEvents.java PRE-CREATION

>   /trunk/giraph/src/main/java/org/apache/giraph/graph/InputSplitPaths.java PRE-CREATION

>   /trunk/giraph/src/main/java/org/apache/giraph/graph/InputSplitsCallable.java 1404751

>   /trunk/giraph/src/main/java/org/apache/giraph/graph/InputSplitsCallableFactory.java
PRE-CREATION 
>   /trunk/giraph/src/main/java/org/apache/giraph/graph/MasterThread.java 1404751 
>   /trunk/giraph/src/main/java/org/apache/giraph/graph/VertexInputFormat.java 1404751

>   /trunk/giraph/src/main/java/org/apache/giraph/graph/VertexInputSplitsCallable.java
PRE-CREATION 
>   /trunk/giraph/src/main/java/org/apache/giraph/graph/VertexInputSplitsCallableFactory.java
PRE-CREATION 
>   /trunk/giraph/src/main/java/org/apache/giraph/graph/VertexReader.java 1404751 
>   /trunk/giraph/src/main/java/org/apache/giraph/graph/VertexResolver.java 1404751 
>   /trunk/giraph/src/main/java/org/apache/giraph/graph/VertexValueInputFormat.java PRE-CREATION

>   /trunk/giraph/src/main/java/org/apache/giraph/graph/VertexValueReader.java PRE-CREATION

>   /trunk/giraph/src/main/java/org/apache/giraph/io/GiraphFileInputFormat.java PRE-CREATION

>   /trunk/giraph/src/main/java/org/apache/giraph/io/GiraphTextInputFormat.java PRE-CREATION

>   /trunk/giraph/src/main/java/org/apache/giraph/io/IntIntTextVertexValueInputFormat.java
PRE-CREATION 
>   /trunk/giraph/src/main/java/org/apache/giraph/io/IntNullTextEdgeInputFormat.java PRE-CREATION

>   /trunk/giraph/src/main/java/org/apache/giraph/io/TextEdgeInputFormat.java PRE-CREATION

>   /trunk/giraph/src/main/java/org/apache/giraph/io/TextVertexInputFormat.java 1404751

>   /trunk/giraph/src/main/java/org/apache/giraph/io/TextVertexValueInputFormat.java PRE-CREATION

>   /trunk/giraph/src/main/java/org/apache/giraph/utils/FileUtils.java 1404751 
>   /trunk/giraph/src/main/java/org/apache/giraph/utils/IntPair.java PRE-CREATION 
>   /trunk/giraph/src/main/java/org/apache/giraph/utils/InternalVertexRunner.java 1404751

>   /trunk/giraph/src/test/java/org/apache/giraph/BspCase.java 1404751 
>   /trunk/giraph/src/test/java/org/apache/giraph/TestEdgeInput.java PRE-CREATION 
>   /trunk/giraph/src/test/java/org/apache/giraph/TestJsonBase64Format.java 1404751 
>   /trunk/giraph/src/test/java/org/apache/giraph/io/TestTextDoubleDoubleAdjacencyListVertexInputFormat.java
1404751 
> 
> Diff: https://reviews.apache.org/r/7784/diff/
> 
> 
> Testing
> -------
> 
> 1) mvn verify
> 2) pseudo-distributed tests
> 3) PageRank on cluster with real text input (will post perf results on the issue)
> 
> 
> Thanks,
> 
> Alessandro Presta
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message