incubator-crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Wills <jwi...@cloudera.com>
Subject Re: Review Request: Adding Sources for NLine and KeyValueText InputFormats
Date Sun, 02 Dec 2012 22:12:12 GMT
Yah, I don't think that RB likes that I rebased relative to the parent
diff. I'm going to close this one and open a new one-- sorry about the spam.


On Sun, Dec 2, 2012 at 2:10 PM, Josh Wills <jwills@cloudera.com> wrote:

>    This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/8215/
>   Review request for crunch.
> By Josh Wills.
>
> *Updated Dec. 2, 2012, 10:10 p.m.*
> Description
>
> Added support for the NLine and KeyValueText InputFormats to the o.a.c.io.text package.
This completes Crunch's support for the InputFormats that ship as part of hadoop-client.
>
> In the process, I refactored the ReaderFactory code that is used to read SequenceFiles
and text files during materialization to eliminate some duplicate code.
>
>   Testing
>
> Integration tests that use the new formats.
>
>   *Bugs: * CRUNCH-119 <https://issues.apache.org/jira/browse/CRUNCH-119>
> Diffs (updated)
>
>    - crunch/src/it/java/org/apache/crunch/io/CompositePathIterableIT.java
>    (796b821)
>    - crunch/src/it/java/org/apache/crunch/io/NLineInputIT.java
>    (PRE-CREATION)
>    - crunch/src/it/java/org/apache/crunch/io/TextFileTableIT.java
>    (PRE-CREATION)
>    - crunch/src/main/java/org/apache/crunch/io/ReadableSource.java
>    (73a13a3)
>    - crunch/src/main/java/org/apache/crunch/io/avro/AvroFileReaderFactory.java
>    (6f21dd2)
>    - crunch/src/main/java/org/apache/crunch/io/avro/AvroFileSource.java
>    (2226556)
>    - crunch/src/main/java/org/apache/crunch/io/impl/AutoClosingIterator.java
>    (d58f290)
>    - crunch/src/main/java/org/apache/crunch/io/impl/FileTableSourceImpl.java
>    (f6e8f1d)
>    - crunch/src/main/java/org/apache/crunch/io/seq/SeqFileReaderFactory.java
>    (ad1b81b)
>    - crunch/src/main/java/org/apache/crunch/io/seq/SeqFileSource.java
>    (e8f3dcf)
>    - crunch/src/main/java/org/apache/crunch/io/seq/SeqFileTableReaderFactory.java
>    (20c749a)
>    - crunch/src/main/java/org/apache/crunch/io/seq/SeqFileTableSource.java
>    (56ed985)
>    - crunch/src/main/java/org/apache/crunch/io/text/LineParser.java
>    (PRE-CREATION)
>    - crunch/src/main/java/org/apache/crunch/io/text/NLineFileSource.java
>    (PRE-CREATION)
>    - crunch/src/main/java/org/apache/crunch/io/text/TextFileReaderFactory.java
>    (a0c48e0)
>    - crunch/src/main/java/org/apache/crunch/io/text/TextFileSource.java
>    (ee51c04)
>    - crunch/src/main/java/org/apache/crunch/io/text/TextFileTableSource.java
>    (PRE-CREATION)
>    - crunch/src/main/java/org/apache/crunch/io/text/TextFileTableSourceTarget.java
>    (PRE-CREATION)
>    - crunch/src/main/java/org/apache/crunch/io/text/TextFileTarget.java
>    (c7e06d3)
>    - crunch/src/test/java/org/apache/crunch/io/avro/AvroFileReaderFactoryTest.java
>    (66863ba)
>
> View Diff <https://reviews.apache.org/r/8215/diff/>
>



-- 
Director of Data Science
Cloudera <http://www.cloudera.com>
Twitter: @josh_wills <http://twitter.com/josh_wills>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message