hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karen Coppage (Jira)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-22006) Hive parquet timestamp compatibility, part 2
Date Wed, 19 Feb 2020 14:50:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-22006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17040113#comment-17040113
] 

Karen Coppage commented on HIVE-22006:
--------------------------------------

[~h-vetinari]

Have you seen [this document|https://docs.google.com/document/d/1gNRww9mZJcHvUDCXklzjFEQGpefsuR_akCDfWsdE35Q]?

Read path for int64 (HIVE-21050, HIVE-21215) is committed to master and write path (HIVE-21216)
should be committed soon.

 

> Hive parquet timestamp compatibility, part 2
> --------------------------------------------
>
>                 Key: HIVE-22006
>                 URL: https://issues.apache.org/jira/browse/HIVE-22006
>             Project: Hive
>          Issue Type: Bug
>    Affects Versions: All Versions
>            Reporter: H. Vetinari
>            Priority: Major
>
> The interaction between HIVE / IMPALA / SPARK writing timestamps is a major source of
headaches in every scenario where such interaction cannot be avoided.
> HIVE-9482 added hive.parquet.timestamp.skip.conversion, which *only* affects the *reading*
of timestamps.
> It formulates the next steps as:
> > Later fix will change the write path to not convert, and stop the read-conversion
even for files written by Hive itself.
> At the very least, HIVE needs a switch to also turn off the conversion on writes. That
would at least allow a setup where all three of HIVE / IMPALA / SPARK can be configured not
to convert on read/write, and can hence safely work on the same data



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message