hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karen Coppage (Jira)" <>
Subject [jira] [Commented] (HIVE-22006) Hive parquet timestamp compatibility, part 2
Date Wed, 19 Feb 2020 14:50:00 GMT


Karen Coppage commented on HIVE-22006:


Have you seen [this document|]?

Read path for int64 (HIVE-21050, HIVE-21215) is committed to master and write path (HIVE-21216)
should be committed soon.


> Hive parquet timestamp compatibility, part 2
> --------------------------------------------
>                 Key: HIVE-22006
>                 URL:
>             Project: Hive
>          Issue Type: Bug
>    Affects Versions: All Versions
>            Reporter: H. Vetinari
>            Priority: Major
> The interaction between HIVE / IMPALA / SPARK writing timestamps is a major source of
headaches in every scenario where such interaction cannot be avoided.
> HIVE-9482 added hive.parquet.timestamp.skip.conversion, which *only* affects the *reading*
of timestamps.
> It formulates the next steps as:
> > Later fix will change the write path to not convert, and stop the read-conversion
even for files written by Hive itself.
> At the very least, HIVE needs a switch to also turn off the conversion on writes. That
would at least allow a setup where all three of HIVE / IMPALA / SPARK can be configured not
to convert on read/write, and can hence safely work on the same data

This message was sent by Atlassian Jira

View raw message