hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Dere (JIRA)" <>
Subject [jira] [Updated] (HIVE-9298) Support reading alternate timestamp formats
Date Wed, 14 Jan 2015 21:45:34 GMT


Jason Dere updated HIVE-9298:
    Attachment: HIVE-9298.1.patch

Initial patch. This adds a new serde property "timestamp.formats" to pass in a comma-delimited
list of alternate timestamp formats.
For example, the following adds 2 alternate formats: yyyy-MM-dd'T'HH:mm:ss, and millis since
Unix epoch, represented by the special case pattern "millis".
ALTER TABLE timestamp_formats SET SERDEPROPERTIES ("timestamp.formats"="yyyy-MM-dd'T'HH:mm:ss,millis");

I'm using Joda date parsing for this item. There are some advantages to using the Joda parsing:
- The fractional seconds parsing for Joda seems to be the only one that looks consistent with
the fractional seconds parsing in the default Timestamp format. For example in Java SimpleDateFormat,
0.1 = 0.001 = 1 millisecond, which is inconsistent with the current Timestamp format.
- The pattern syntax is mostly compatible with SimpleDateFormat
- It supports adding multiple date formats
- It is extensible enough to add a custom parser to parse numeric values as millis since Unix
epoch (for HIVE-3844).
- It's not perfect either: Joda DateTime only supports up to millisecond precision, so a custom
timestamp format will only be able to support up to millis precision as opposed to the default
Timestamp format which can support nanosecond precision.

> Support reading alternate timestamp formats
> -------------------------------------------
>                 Key: HIVE-9298
>                 URL:
>             Project: Hive
>          Issue Type: Bug
>          Components: Serializers/Deserializers
>            Reporter: Jason Dere
>            Assignee: Jason Dere
>         Attachments: HIVE-9298.1.patch
> There are some users who want to be able to parse ISO-8601 timestamps, as well to set
their own custom timestamp formats. We may be able to support this in LazySimpleSerDe through
the use of a SerDe parameter to specify one or more alternative timestamp patterns to use
to parse timestamp values from string.
> If we are doing this it might also be nice to work in support for HIVE-3844, to parse
numeric strings as timestamp by treating the numeric value as millis since Unix epoch. This
can be enabled through the SerDe params as well.

This message was sent by Atlassian JIRA

View raw message