giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nitay Joffe" <ni...@apache.org>
Subject Re: Review Request: GIRAPH-588: More flexible Hive input
Date Wed, 27 Mar 2013 23:56:21 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10126/#review18450
-----------------------------------------------------------

Ship it!


Ship It!

- Nitay Joffe


On March 27, 2013, 4:19 p.m., Maja Kabiljo wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/10126/
> -----------------------------------------------------------
> 
> (Updated March 27, 2013, 4:19 p.m.)
> 
> 
> Review request for giraph and Nitay Joffe.
> 
> 
> Description
> -------
> 
> Currently with Hive input formats it's only possible to convert single row to single
vertex/edge. We should support some rows to be skipped. General multiple rows to multiple
vertices/edges conversion can be useful (one example - we have edge input, sorted by source
id, and we want to keep only top k weighted edges per vertex).
> 
> Added some wrappers for readers to convert them to Iterator-like classes.
> 
> 
> This addresses bug GIRAPH-588.
>     https://issues.apache.org/jira/browse/GIRAPH-588
> 
> 
> Diffs
> -----
> 
>   giraph-core/src/main/java/org/apache/giraph/io/iterables/EdgeReaderWrapper.java PRE-CREATION

>   giraph-core/src/main/java/org/apache/giraph/io/iterables/EdgeWithSource.java PRE-CREATION

>   giraph-core/src/main/java/org/apache/giraph/io/iterables/GiraphReader.java PRE-CREATION

>   giraph-core/src/main/java/org/apache/giraph/io/iterables/IteratorToReaderWrapper.java
PRE-CREATION 
>   giraph-core/src/main/java/org/apache/giraph/io/iterables/VertexReaderWrapper.java PRE-CREATION

>   giraph-core/src/main/java/org/apache/giraph/io/iterables/package-info.java PRE-CREATION

>   giraph-hive/src/main/java/org/apache/giraph/hive/HiveGiraphRunner.java efc08d3 
>   giraph-hive/src/main/java/org/apache/giraph/hive/common/DefaultConfigurableAndTableSchemaAware.java
PRE-CREATION 
>   giraph-hive/src/main/java/org/apache/giraph/hive/input/RecordReaderWrapper.java PRE-CREATION

>   giraph-hive/src/main/java/org/apache/giraph/hive/input/edge/AbstractHiveToEdge.java
f29fea7 
>   giraph-hive/src/main/java/org/apache/giraph/hive/input/edge/HiveEdgeInputFormat.java
18b40c2 
>   giraph-hive/src/main/java/org/apache/giraph/hive/input/edge/HiveEdgeReader.java 6fb183a

>   giraph-hive/src/main/java/org/apache/giraph/hive/input/edge/HiveToEdge.java 2205b82

>   giraph-hive/src/main/java/org/apache/giraph/hive/input/edge/SimpleHiveToEdge.java PRE-CREATION

>   giraph-hive/src/main/java/org/apache/giraph/hive/input/vertex/AbstractHiveToVertex.java
PRE-CREATION 
>   giraph-hive/src/main/java/org/apache/giraph/hive/input/vertex/AbstractHiveToVertexEdges.java
d0668f6 
>   giraph-hive/src/main/java/org/apache/giraph/hive/input/vertex/AbstractHiveToVertexValue.java
9ab316f 
>   giraph-hive/src/main/java/org/apache/giraph/hive/input/vertex/HiveToVertex.java PRE-CREATION

>   giraph-hive/src/main/java/org/apache/giraph/hive/input/vertex/HiveToVertexEdges.java
8076a8a 
>   giraph-hive/src/main/java/org/apache/giraph/hive/input/vertex/HiveToVertexValue.java
593eb9a 
>   giraph-hive/src/main/java/org/apache/giraph/hive/input/vertex/HiveVertexInputFormat.java
25c7a26 
>   giraph-hive/src/main/java/org/apache/giraph/hive/input/vertex/HiveVertexReader.java
541176f 
>   giraph-hive/src/main/java/org/apache/giraph/hive/input/vertex/SimpleHiveToVertex.java
PRE-CREATION 
>   giraph-hive/src/main/java/org/apache/giraph/hive/input/vertex/SimpleNoEdgesHiveToVertex.java
PRE-CREATION 
>   giraph-hive/src/main/java/org/apache/giraph/hive/output/AbstractVertexToHive.java 8e3f1ca

> 
> Diff: https://reviews.apache.org/r/10126/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Maja Kabiljo
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message