Mailing-List: contact dev-help@giraph.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@giraph.apache.org
Content-Type: multipart/alternative;
 boundary="===============7702315013805625830=="
MIME-Version: 1.0
Subject: Re: Review Request 13756: GIRAPH-732: EdgeOutputFormat API
From: "Armando" <a.miraglia@student.vu.nl>
To: "Armando" <a.miraglia@student.vu.nl>,
 "Claudio Martella" <claudio.martella@gmail.com>,
 "Nitay Joffe" <nitay@apache.org>
Cc: "giraph" <dev@giraph.apache.org>
Date: Mon, 26 Aug 2013 11:04:26 -0000
Message-ID: <20130826110426.16847.34604@reviews.apache.org>
Auto-Submitted: auto-generated
Sender: "Armando" <noreply@reviews.apache.org>
References: <20130825203055.16846.57321@reviews.apache.org>
In-Reply-To: <20130825203055.16846.57321@reviews.apache.org>
Reply-To: "Armando" <a.miraglia@student.vu.nl>

--===============7702315013805625830==
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit


> On Aug. 25, 2013, 8:30 p.m., Nitay Joffe wrote:
> > giraph-core/src/main/java/org/apache/giraph/worker/BspServiceWorker.java, line 1099
> > <https://reviews.apache.org/r/13756/diff/2/?file=344112#file344112line1099>
> >
> >     Should we track and add something in here about number of edges saved instead of just vertices, seeing as this is all about edge writing?

something like this is acceptable?

LoggerUtils.setStatusAndLog(getContext(), LOG, Level.INFO,         
                        "saveEdges: Saved " + edges +                                  
                        " edges out of " + partition.getEdgeCount() +                  
                        " partition edges, on partition " + partitionIndex +           
                        " out of " + numPartitions);


- Armando


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13756/#review25523
-----------------------------------------------------------


On Aug. 23, 2013, 1:34 a.m., Armando wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/13756/
> -----------------------------------------------------------
> 
> (Updated Aug. 23, 2013, 1:34 a.m.)
> 
> 
> Review request for giraph, Armando, Claudio Martella, and Nitay Joffe.
> 
> 
> Bugs: GIRAPH-732
>     https://issues.apache.org/jira/browse/GIRAPH-732
> 
> 
> Repository: giraph-git
> 
> 
> Description
> -------
> 
> This patch is a possible solution to provide an EdgeOutputFormat API in Giraph.
> The idea is to add the same logic as the one for the vertices for edges. For this reason I add saveEdges after saveVertices.
> This requires the user to have to different paths for the resulting files. For this reason I have implemented TextEdgeOutputFormat
> and modified TextVertexOutputFormat so that a subdirectory can be specified.
> The code I provide should be compliant with previous versions of hadoop as well as YARN (I was yet not able to test it on YARN).
> It is also retro-compatible since when not specified, TextVertexOutputFormat behaves as before.
> I am also providing an actual usable implementation with the associated tests (SrcIdDstIdEdgeValueOutputFormat).
> Also very interesting is the fact that this implementation is totally transparent to the classes implementing TextVertexOutputFormat.
> 
> passed "maven verify"
> 
> 
> Diffs
> -----
> 
>   giraph-core/src/main/java/org/apache/giraph/GiraphRunner.java 1bd79b5 
>   giraph-core/src/main/java/org/apache/giraph/conf/GiraphClasses.java 71fe885 
>   giraph-core/src/main/java/org/apache/giraph/conf/GiraphConfiguration.java 23bcd32 
>   giraph-core/src/main/java/org/apache/giraph/conf/GiraphConstants.java c276c2a 
>   giraph-core/src/main/java/org/apache/giraph/conf/ImmutableClassesGiraphConfiguration.java 49a2ebc 
>   giraph-core/src/main/java/org/apache/giraph/io/EdgeOutputFormat.java PRE-CREATION 
>   giraph-core/src/main/java/org/apache/giraph/io/EdgeWriter.java PRE-CREATION 
>   giraph-core/src/main/java/org/apache/giraph/io/formats/GiraphTextOutputFormat.java PRE-CREATION 
>   giraph-core/src/main/java/org/apache/giraph/io/formats/SrcIdDstIdEdgeValueTextOutputFormat.java PRE-CREATION 
>   giraph-core/src/main/java/org/apache/giraph/io/formats/TextEdgeOutputFormat.java PRE-CREATION 
>   giraph-core/src/main/java/org/apache/giraph/io/formats/TextVertexOutputFormat.java c91d543 
>   giraph-core/src/main/java/org/apache/giraph/io/internal/WrappedEdgeOutputFormat.java PRE-CREATION 
>   giraph-core/src/main/java/org/apache/giraph/utils/ConfigurationUtils.java 745764b 
>   giraph-core/src/main/java/org/apache/giraph/worker/BspServiceWorker.java da1e7fb 
>   giraph-core/src/test/java/org/apache/giraph/io/TestSrcIdDstIdEdgeValueTextOutputFormat.java PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/13756/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Armando
> 
>


--===============7702315013805625830==--