giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Claudio Martella" <claudio.marte...@gmail.com>
Subject Re: Review Request 13756: GIRAPH-732: EdgeOutputFormat API
Date Mon, 26 Aug 2013 10:54:28 GMT


> On Aug. 25, 2013, 8:30 p.m., Nitay Joffe wrote:
> > giraph-core/src/main/java/org/apache/giraph/io/formats/SrcIdDstIdEdgeValueTextOutputFormat.java,
line 79
> > <https://reviews.apache.org/r/13756/diff/2/?file=344107#file344107line79>
> >
> >     what is the use case for this reversed output?
> 
> Armando wrote:
>     Honestly, I don't know. Inspecting the code of IdWithValueTextOutputFormat I assumed
that having that use-case could be useful to someone. Can be dropped, just let me know

I also wondered about the uses cases, but I don't think it will make any damage, but potentially
could help people who'd find one.


- Claudio


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13756/#review25523
-----------------------------------------------------------


On Aug. 23, 2013, 1:34 a.m., Armando wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/13756/
> -----------------------------------------------------------
> 
> (Updated Aug. 23, 2013, 1:34 a.m.)
> 
> 
> Review request for giraph, Armando, Claudio Martella, and Nitay Joffe.
> 
> 
> Bugs: GIRAPH-732
>     https://issues.apache.org/jira/browse/GIRAPH-732
> 
> 
> Repository: giraph-git
> 
> 
> Description
> -------
> 
> This patch is a possible solution to provide an EdgeOutputFormat API in Giraph.
> The idea is to add the same logic as the one for the vertices for edges. For this reason
I add saveEdges after saveVertices.
> This requires the user to have to different paths for the resulting files. For this reason
I have implemented TextEdgeOutputFormat
> and modified TextVertexOutputFormat so that a subdirectory can be specified.
> The code I provide should be compliant with previous versions of hadoop as well as YARN
(I was yet not able to test it on YARN).
> It is also retro-compatible since when not specified, TextVertexOutputFormat behaves
as before.
> I am also providing an actual usable implementation with the associated tests (SrcIdDstIdEdgeValueOutputFormat).
> Also very interesting is the fact that this implementation is totally transparent to
the classes implementing TextVertexOutputFormat.
> 
> passed "maven verify"
> 
> 
> Diffs
> -----
> 
>   giraph-core/src/main/java/org/apache/giraph/GiraphRunner.java 1bd79b5 
>   giraph-core/src/main/java/org/apache/giraph/conf/GiraphClasses.java 71fe885 
>   giraph-core/src/main/java/org/apache/giraph/conf/GiraphConfiguration.java 23bcd32 
>   giraph-core/src/main/java/org/apache/giraph/conf/GiraphConstants.java c276c2a 
>   giraph-core/src/main/java/org/apache/giraph/conf/ImmutableClassesGiraphConfiguration.java
49a2ebc 
>   giraph-core/src/main/java/org/apache/giraph/io/EdgeOutputFormat.java PRE-CREATION 
>   giraph-core/src/main/java/org/apache/giraph/io/EdgeWriter.java PRE-CREATION 
>   giraph-core/src/main/java/org/apache/giraph/io/formats/GiraphTextOutputFormat.java
PRE-CREATION 
>   giraph-core/src/main/java/org/apache/giraph/io/formats/SrcIdDstIdEdgeValueTextOutputFormat.java
PRE-CREATION 
>   giraph-core/src/main/java/org/apache/giraph/io/formats/TextEdgeOutputFormat.java PRE-CREATION

>   giraph-core/src/main/java/org/apache/giraph/io/formats/TextVertexOutputFormat.java
c91d543 
>   giraph-core/src/main/java/org/apache/giraph/io/internal/WrappedEdgeOutputFormat.java
PRE-CREATION 
>   giraph-core/src/main/java/org/apache/giraph/utils/ConfigurationUtils.java 745764b 
>   giraph-core/src/main/java/org/apache/giraph/worker/BspServiceWorker.java da1e7fb 
>   giraph-core/src/test/java/org/apache/giraph/io/TestSrcIdDstIdEdgeValueTextOutputFormat.java
PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/13756/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Armando
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message