hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "justin coffey" <j.cof...@criteo.com>
Subject Re: Review Request 22174: HIVE-6394 Implement Timestmap in ParquetSerde
Date Thu, 05 Jun 2014 08:43:51 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22174/#review44805
-----------------------------------------------------------



ql/src/java/org/apache/hadoop/hive/ql/io/parquet/convert/ETypeConverter.java
<https://reviews.apache.org/r/22174/#comment79343>

    A stupid question perhaps, but is INT96 reserved for timestamps in parquet?
    
    I dug this up, but not sure if it's definitive: https://github.com/Parquet/parquet-mr/issues/101


- justin coffey


On June 5, 2014, 7:33 a.m., Szehon Ho wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/22174/
> -----------------------------------------------------------
> 
> (Updated June 5, 2014, 7:33 a.m.)
> 
> 
> Review request for hive, Brock Noland, justin coffey, and Xuefu Zhang.
> 
> 
> Bugs: HIVE-6394
>     https://issues.apache.org/jira/browse/HIVE-6394
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> This uses the Jodd library to convert java.sql.Timestamp type used by Hive into the {julian-day:nanos}
format expected by parquet, and vice-versa.
> 
> 
> Diffs
> -----
> 
>   data/files/parquet_types.txt 0be390b 
>   pom.xml 4bb8880 
>   ql/pom.xml 13c477a 
>   ql/src/java/org/apache/hadoop/hive/ql/io/parquet/convert/ETypeConverter.java 4da0d30

>   ql/src/java/org/apache/hadoop/hive/ql/io/parquet/convert/HiveSchemaConverter.java 29f7e11

>   ql/src/java/org/apache/hadoop/hive/ql/io/parquet/serde/ArrayWritableObjectInspector.java
57161d8 
>   ql/src/java/org/apache/hadoop/hive/ql/io/parquet/serde/ParquetHiveSerDe.java fb2f5a8

>   ql/src/java/org/apache/hadoop/hive/ql/io/parquet/utils/NanoTimeUtils.java PRE-CREATION

>   ql/src/java/org/apache/hadoop/hive/ql/io/parquet/write/DataWritableWriter.java 3490061

>   ql/src/test/org/apache/hadoop/hive/ql/io/parquet/serde/TestParquetTimestampUtils.java
PRE-CREATION 
>   ql/src/test/queries/clientpositive/parquet_types.q 5d6333c 
>   ql/src/test/results/clientpositive/parquet_types.q.out c23f7f1 
> 
> Diff: https://reviews.apache.org/r/22174/diff/
> 
> 
> Testing
> -------
> 
> Unit tests the new libraries, and also added timestamp data in the "parquet_types" q-test.
> 
> 
> Thanks,
> 
> Szehon Ho
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message