hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rui Li (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-9482) Hive parquet timestamp compatibility
Date Tue, 26 Jul 2016 01:57:20 GMT

    [ https://issues.apache.org/jira/browse/HIVE-9482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15393035#comment-15393035
] 

Rui Li commented on HIVE-9482:
------------------------------

Hi [~szehon], is there a follow on task for the write path?

> Hive parquet timestamp compatibility
> ------------------------------------
>
>                 Key: HIVE-9482
>                 URL: https://issues.apache.org/jira/browse/HIVE-9482
>             Project: Hive
>          Issue Type: Bug
>          Components: File Formats
>    Affects Versions: 0.15.0
>            Reporter: Szehon Ho
>            Assignee: Szehon Ho
>             Fix For: 1.2.0
>
>         Attachments: HIVE-9482.2.patch, HIVE-9482.patch, HIVE-9482.patch, parquet_external_time.parq
>
>
> In current Hive implementation, timestamps are stored in UTC (converted from current
timezone), based on original parquet timestamp spec.
> However, we find this is not compatibility with other tools, and after some investigation
it is not the way of the other file formats, or even some databases (Hive Timestamp is more
equivalent of 'timestamp without timezone' datatype).
> This is the first part of the fix, which will restore compatibility with parquet-timestamp
files generated by external tools by skipping conversion on reading.
> Later fix will change the write path to not convert, and stop the read-conversion even
for files written by Hive itself.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message