hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dong Chen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-10642) Update Timestamp type mapping to Parquet
Date Thu, 07 May 2015 16:37:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-10642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14532962#comment-14532962
] 

Dong Chen commented on HIVE-10642:
----------------------------------

And with new support, generated timestamp value in Parquet file will have less precise.

If users' raw data is nano precise, and they want to keep the precise when saved as Parquet
file, maybe old support still have to be used. How about we add a conf param and give user
choice?

> Update Timestamp type mapping to Parquet 
> -----------------------------------------
>
>                 Key: HIVE-10642
>                 URL: https://issues.apache.org/jira/browse/HIVE-10642
>             Project: Hive
>          Issue Type: Sub-task
>            Reporter: Dong Chen
>            Assignee: Dong Chen
>         Attachments: HIVE-10642-parquet.patch
>
>
> Timestamp in Hive was originally mapped to INT_96 in Parquet before. And inside Parquet
it use Binary to save values.
> When working on HIVE-10255, I found predicate with timestamp does not work, since comparing
Binary representative of Timestamp value can not give correct relationship.
> Meanwhile, in latest version Parquet, the timestamp is mapped to INT_64. Refer to https://github.com/Parquet/parquet-format/blob/master/LogicalTypes.md
> Therefore, this Jira will update the mapping of Timestamp type in Hive, and keep it backward
compatible.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message