avro-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dmitry Kovalev (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AVRO-739) Add Date/Time data types
Date Thu, 07 Aug 2014 18:04:14 GMT

    [ https://issues.apache.org/jira/browse/AVRO-739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14089534#comment-14089534

Dmitry Kovalev commented on AVRO-739:

Attaching a first draft - please review. My comments/issues:

- in its current form, the spec only provides for a precision of up to a millisecond - this
may save space and be a most universally used precision, but modern platforms and ISO standard
provide for better precision

- we could support high-precision time as a separate type on the grounds that it is less frequently
used, or we could introduce an optional "precision" annotation which will say define a number
of decimal places in second fractions

- also, whether we only support the millis or also a higher precision, in either case your
reasoning about simple names implying "canonical" use would arguably suggest smth like "timestamp"
instead of "timestamp-millis" and "time" instead of "time-millis"? Was there a specific reason
for adding "millis" in Parquet and is it important from interop point of view if Avro adopts
a different name (as long as the actual definition is the same)?

- I didn't provide a binary representation for Timestamp-timezone as I'm not entirely sure
how it would look like and whether it will be popular at all, compared to the string representation

- finally, re Parquet Interval type - I used to think (and ISO, Noda time etc seem to agree)
that an "interval" means an interval on a global timeline, i.e. something with start and end
at a specific instant in time, whereas what the current wording defines is actually better
called a Duration. So the question is again - whether there was a specific reason to call
it Interval in Parquet and if naming it Duration in Avro impacts Hadoop interop?

> Add Date/Time data types
> ------------------------
>                 Key: AVRO-739
>                 URL: https://issues.apache.org/jira/browse/AVRO-739
>             Project: Avro
>          Issue Type: New Feature
>          Components: spec
>            Reporter: Jeff Hammerbacher
>             Fix For: 1.7.8
>         Attachments: AVRO-739-datetime-spec.xml.patch, AVRO-739.patch

This message was sent by Atlassian JIRA

View raw message