arrow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Wes McKinney <wesmck...@gmail.com>
Subject Re: Making some decisions about date and time types
Date Fri, 17 Mar 2017 21:25:52 GMT
> Am I correct that timestamp is a 64 bit signed integer representing microseconds since
1970? If so, it would be helpful to state the minimum and maximum values in the spec.

Timestamps have a unit from SECOND to NANOSECOND, but otherwise they
are stored as a 64-bit signed integer. I agree it would be good to put
the minimum and maximum values in the spec

On Fri, Mar 17, 2017 at 4:23 PM, Julian Hyde <jhyde@apache.org> wrote:
> Am I correct that timestamp is a 64 bit signed integer representing microseconds since
1970? If so, it would be helpful to state the minimum and maximum values in the spec.
>
> I can’t quite imagine a use case for microsecond time, given that it takes the same
number of bits as a timestamp. But still, no harm in including it.
>
> At some point someone will want MONTH as a time unit (to support SQL’s year-to-month
interval type) and someone will want nanosecond timestamp (problematic, because it needs more
than 64 bits for a useful range to dates). But these can wait until version 2.
>
> Julian
>
>
>> On Mar 17, 2017, at 9:51 AM, Wes McKinney <wesmckinn@gmail.com> wrote:
>>
>> hi folks,
>>
>> We have some format decisions to make about all 3 of the primary
>> temporal types in Arrow:
>>
>> ARROW-617 - Time type
>> - It is proposed to add the type bit width to the metadata for
>> clarity, and using the smallest type that can accommodate a particular
>> time unit
>> - PATCH: https://github.com/apache/arrow/pull/385
>>
>> ARROW-316: Date type
>> - It is proposed to add a DateUnit to indicate day-based date (a la
>> PostgreSQL and other systems) as int32 vs. millisecond-based date as
>> int64 (a la Joda, and current Arrow Java)
>> - PATCH: https://github.com/apache/arrow/pull/390
>>
>> ARROW-637: Timestamp type
>> - It is proposed to add a timezone string to the metadata as to
>> disambiguate TZ-naive vs. TZ-aware data, but otherwise display only
>> (changing the time zone does not alter the physical int64 timestamp
>> values)
>> - PATCH: https://github.com/apache/arrow/pull/388
>>
>> There seems to be some degree of consensus on all 3 of these, but it
>> would be good to reach a final decision and merge patches so that we
>> can do the corresponding dev work in Java and C++, and hopefully get
>> integration tests working in time for the Arrow 0.3 release.
>>
>> Thanks!
>> Wes
>

Mime
View raw message