hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ryan Blue" <b...@apache.org>
Subject Re: Review Request 30717: HIVE-8119: Implement Date in ParquetSerde
Date Tue, 10 Feb 2015 19:50:01 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30717/#review71844
-----------------------------------------------------------


One minor thing, but this looks good otherwise. Sergio knows more about the compatibility
between this and his recent performance work and the object inspector code, so I'll leave
that to him for review.


ql/src/java/org/apache/hadoop/hive/ql/io/parquet/convert/HiveSchemaConverter.java
<https://reviews.apache.org/r/30717/#comment117710>

    For primitive types, this should be using the Types API (like the line above) because
we're going to remove the constructors from the public API in favor of the bulider. This is
to avoid invalid types, like an INT64 with a DATE annotation.
    
    This should be:
    ```java
    Types.primitive(repetition, INT32).as(DATE).named(name);
    ```


- Ryan Blue


On Feb. 5, 2015, 11:51 p.m., Dong Chen wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30717/
> -----------------------------------------------------------
> 
> (Updated Feb. 5, 2015, 11:51 p.m.)
> 
> 
> Review request for hive.
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> HIVE-8119: Implement Date in ParquetSerde
> 
> This patch map the Date in Hive to INT32 in Parquet, based on the Parquet Logical Type
Definitions in https://github.com/apache/incubator-parquet-format/blob/master/LogicalTypes.md
> 
> 
> Diffs
> -----
> 
>   data/files/parquet_types.txt 31a10c9 
>   ql/src/java/org/apache/hadoop/hive/ql/io/parquet/convert/ETypeConverter.java 377e362

>   ql/src/java/org/apache/hadoop/hive/ql/io/parquet/convert/HiveSchemaConverter.java e5bd70c

>   ql/src/java/org/apache/hadoop/hive/ql/io/parquet/serde/ArrayWritableObjectInspector.java
bb066af 
>   ql/src/java/org/apache/hadoop/hive/ql/io/parquet/serde/ParquetHiveSerDe.java 9199127

>   ql/src/java/org/apache/hadoop/hive/ql/io/parquet/write/DataWritableWriter.java 1d83bf3

>   ql/src/test/org/apache/hadoop/hive/ql/io/parquet/TestHiveSchemaConverter.java f232c57

>   ql/src/test/queries/clientnegative/parquet_date.q 89d3602 
>   ql/src/test/queries/clientpositive/parquet_types.q 806db24 
>   ql/src/test/results/clientnegative/parquet_date.q.out d1c38d6 
>   ql/src/test/results/clientpositive/parquet_types.q.out dc5ceb0 
> 
> Diff: https://reviews.apache.org/r/30717/diff/
> 
> 
> Testing
> -------
> 
> UT passed. 2 tests are added
> 
> 
> Thanks,
> 
> Dong Chen
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message