hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yang Yang (JIRA)" <>
Subject [jira] [Commented] (HIVE-6394) Implement Timestmap in ParquetSerde
Date Tue, 03 Feb 2015 06:37:34 GMT


Yang Yang commented on HIVE-6394:

the parquet spec about logical types and Timestamp specifically, seems to say
"TIMESTAMP_MILLIS is used for a combined logical date and time type. It must annotate an int64
that stores the number of milliseconds from the Unix epoch, 00:00:00.000 on 1 January 1970,


i.e. here it says that the type is only precise to the point of miliseconds and it starts
from 1970.

but if u look at the hive-parquet code in
it seems that hive's encoding of timestamp on parquet is of a different spec, precise to the
point of nano seconds, and starting from "Monday, January 1, 4713 " (defined in jodd.datetime.JDateTime)

so Hive's parquet timestamp storage is completely different from the above spec ?

> Implement Timestmap in ParquetSerde
> -----------------------------------
>                 Key: HIVE-6394
>                 URL:
>             Project: Hive
>          Issue Type: Sub-task
>          Components: Serializers/Deserializers
>            Reporter: Jarek Jarcec Cecho
>            Assignee: Szehon Ho
>              Labels: Parquet
>             Fix For: 0.14.0
>         Attachments: HIVE-6394.2.patch, HIVE-6394.3.patch, HIVE-6394.4.patch, HIVE-6394.5.patch,
HIVE-6394.6.patch, HIVE-6394.6.patch, HIVE-6394.7.patch, HIVE-6394.patch
> This JIRA is to implement timestamp support in Parquet SerDe.

This message was sent by Atlassian JIRA

View raw message