avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ron Bodkin <rbod...@thinkbiganalytics.com>
Subject Re: Feature for Date/Time Data Types in Avro?
Date Tue, 18 Jan 2011 18:38:44 GMT
Overall, yes. A couple of points worth addressing in a design:

1) Do we want to allow encoding time zone data in the records? Storing a
raw timestamp is sometimes not ideal. It's worth looking at how SQL allows
timestamps with and without time zones. Is that simpler, or is it actually
more complex?
2) Do we want to allow dates (for storing a day, without a timestamp)?
3) It would be nice to allow some flexibility in the implementation
classes for dates, e.g., letting Java users use Joda time classes as well
as java.util.Date

Ron


Ron Bodkin
CEO
Think Big Analytics
m: +1 (415) 509-2895








On 1/18/11 8:42 AM, "Doug Cutting" <cutting@apache.org> wrote:

>The way that I have imagined doing this is to specify a standard schema
>for dates, then implementations can optionally map this to a native date
>type.
>
>The schema could be a record containing a long, e.g.:
>
>{"type": "record", "name":"org.apache.avro.lib.Date", "fields" : [
>   {"name": "time", "type": "long"}
>  ]
>}
>
>Java could read this into a java.util.Date, Python to a datetime, etc.
>Such conventions could be added to the Avro specification.
>
>Does this sound like a reasonable approach?
>
>Doug
>
>On 01/17/2011 05:54 PM, Ron Bodkin wrote:
>> Has anyone discussed the possibility of having built-in support for a
>> date/time stamp data type in Avro? I think it'd be helpful, since dates
>> and timestamps are often used as keys in processing map/reduce data (and
>> in RPC systems). It's unpleasant to have to write code that converts
>> longs or strings into dates or timestamps. Minimally, it would be useful
>> to allow generating date/time stamps from long timestamps in the client
>> APIs various language code and to have support for working with Dates in
>> the Java reflection API.
>>
>> I'd like to get feedback from others if they'd also like to see support
>> for date/time data types in Avro. It seems like a generally useful
>> feature that would be worth adding with a patch.
>>
>> Thanks,
>> Ron
>>
>> Ron Bodkin
>> CEO
>> Think Big Analytics
>> m: +1 (415) 509-2895
>>
>>



Mime
View raw message