giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alessandro Presta" <alessan...@fb.com>
Subject Review Request: Edge-based input from HCatalog
Date Tue, 13 Nov 2012 03:46:14 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/8034/
-----------------------------------------------------------

Review request for giraph.


Description
-------

Implemented Edge/VertexValue input formats for HCatalog.
Unfortunately, I had to pretty much copy some functionality from HCatalog because of privacy
restrictions, and add our own GiraphHCatInputFormat (much like with GiraphFileInputFormat).
Also, I had to make HiveGiraphRunner a little less type-safe, because HCatalogVertexValueInputFormat
is not a subclass of HCatalogVertexInputFormat. If this is an issue, I can add an empty interface
to address it.


This addresses bug GIRAPH-405.
    https://issues.apache.org/jira/browse/GIRAPH-405


Diffs
-----

  /trunk/giraph-formats-contrib/pom.xml 1406239 
  /trunk/giraph-formats-contrib/src/main/java/org/apache/giraph/io/hcatalog/GiraphHCatInputFormat.java
PRE-CREATION 
  /trunk/giraph-formats-contrib/src/main/java/org/apache/giraph/io/hcatalog/HCatalogEdgeInputFormat.java
PRE-CREATION 
  /trunk/giraph-formats-contrib/src/main/java/org/apache/giraph/io/hcatalog/HCatalogVertexInputFormat.java
1406239 
  /trunk/giraph-formats-contrib/src/main/java/org/apache/giraph/io/hcatalog/HCatalogVertexOutputFormat.java
1406239 
  /trunk/giraph-formats-contrib/src/main/java/org/apache/giraph/io/hcatalog/HCatalogVertexValueInputFormat.java
PRE-CREATION 
  /trunk/giraph-formats-contrib/src/main/java/org/apache/giraph/io/hcatalog/HiveGiraphRunner.java
1406239 
  /trunk/giraph-formats-contrib/src/main/java/org/apache/hcatalog/mapreduce/HCatUtils.java
PRE-CREATION 
  /trunk/giraph-formats-contrib/src/main/java/org/apache/hcatalog/mapreduce/package-info.java
PRE-CREATION 
  /trunk/pom.xml 1406239 

Diff: https://reviews.apache.org/r/8034/diff/


Testing
-------

- mvn verify
- tested on a real application that runs on top of Hive


Thanks,

Alessandro Presta


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message