giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nitay Joffe" <ni...@apache.org>
Subject Re: Review Request: Edge-based input from HCatalog
Date Mon, 19 Nov 2012 21:30:49 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/8034/#review13587
-----------------------------------------------------------



/trunk/giraph-formats-contrib/src/main/java/org/apache/giraph/io/hcatalog/GiraphHCatInputFormat.java
<https://reviews.apache.org/r/8034/#comment29156>

    yeah it's really unfortunate we have to do this - but hopefully sooner it will be cleaner
as things progress



/trunk/giraph-formats-contrib/src/main/java/org/apache/giraph/io/hcatalog/GiraphHCatInputFormat.java
<https://reviews.apache.org/r/8034/#comment29154>

    why do you need this and the method above, just have the Configuration one alone.



/trunk/giraph-formats-contrib/src/main/java/org/apache/hcatalog/mapreduce/HCatUtils.java
<https://reviews.apache.org/r/8034/#comment29158>

    Can we make all the parts of this diff where you copy code into TODOs and file JIRAs with
HCatalog to make the appropriate changes? We should track that these are copied over and work
towards remove them as opposed to letting them pollute our code.



/trunk/pom.xml
<https://reviews.apache.org/r/8034/#comment29153>

    I think I do this in one of my diffs, in case not - can you make some variable like hive.version
to use here everywhere?


- Nitay Joffe


On Nov. 13, 2012, 3:46 a.m., Alessandro Presta wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/8034/
> -----------------------------------------------------------
> 
> (Updated Nov. 13, 2012, 3:46 a.m.)
> 
> 
> Review request for giraph.
> 
> 
> Description
> -------
> 
> Implemented Edge/VertexValue input formats for HCatalog.
> Unfortunately, I had to pretty much copy some functionality from HCatalog because of
privacy restrictions, and add our own GiraphHCatInputFormat (much like with GiraphFileInputFormat).
> Also, I had to make HiveGiraphRunner a little less type-safe, because HCatalogVertexValueInputFormat
is not a subclass of HCatalogVertexInputFormat. If this is an issue, I can add an empty interface
to address it.
> 
> 
> This addresses bug GIRAPH-405.
>     https://issues.apache.org/jira/browse/GIRAPH-405
> 
> 
> Diffs
> -----
> 
>   /trunk/giraph-formats-contrib/pom.xml 1406239 
>   /trunk/giraph-formats-contrib/src/main/java/org/apache/giraph/io/hcatalog/GiraphHCatInputFormat.java
PRE-CREATION 
>   /trunk/giraph-formats-contrib/src/main/java/org/apache/giraph/io/hcatalog/HCatalogEdgeInputFormat.java
PRE-CREATION 
>   /trunk/giraph-formats-contrib/src/main/java/org/apache/giraph/io/hcatalog/HCatalogVertexInputFormat.java
1406239 
>   /trunk/giraph-formats-contrib/src/main/java/org/apache/giraph/io/hcatalog/HCatalogVertexOutputFormat.java
1406239 
>   /trunk/giraph-formats-contrib/src/main/java/org/apache/giraph/io/hcatalog/HCatalogVertexValueInputFormat.java
PRE-CREATION 
>   /trunk/giraph-formats-contrib/src/main/java/org/apache/giraph/io/hcatalog/HiveGiraphRunner.java
1406239 
>   /trunk/giraph-formats-contrib/src/main/java/org/apache/hcatalog/mapreduce/HCatUtils.java
PRE-CREATION 
>   /trunk/giraph-formats-contrib/src/main/java/org/apache/hcatalog/mapreduce/package-info.java
PRE-CREATION 
>   /trunk/pom.xml 1406239 
> 
> Diff: https://reviews.apache.org/r/8034/diff/
> 
> 
> Testing
> -------
> 
> - mvn verify
> - tested on a real application that runs on top of Hive
> 
> 
> Thanks,
> 
> Alessandro Presta
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message