avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stu Hood" <stu.h...@rackspace.com>
Subject Re: java specific implementation uses GenericArray ?
Date Fri, 27 Aug 2010 17:19:02 GMT
Hey Patrick,

Have you gotten any time to work on this?

Thanks,
Stu

-----Original Message-----
From: "Patrick Linehan" <plinehan@plinehan.com>
Sent: Wednesday, August 18, 2010 5:37pm
To: user@avro.apache.org
Subject: Re: java specific implementation uses GenericArray ?

On Wed, Aug 18, 2010 at 11:23 AM, Doug Cutting <cutting@apache.org> wrote:

> On 08/16/2010 02:46 PM, Patrick Linehan wrote:
>
>> does anyone have any suggestions for dealing with large lists/arrays of
>> primitive values in avro?
>>
>> in my case (numerical algorithms), my naive mapping of a vector type
>> (mathematical vectors, not java Vectors) to an avro specific type
>> generates a GenericArray<Double>.  needless to say, i would prefer to
>> avoid the cost of boxing up all the individual floating point numbers.
>>
>> is it possible to coerce avro into using raw java primitive arrays, e.g.
>> "double[]"?
>>
>
> It should be possible to subclass things to effect this.  In particular:
>
>  - extend GenericData, overriding isArray()
>  - extend GenericDatumReader, overriding readArray()
>  - extend GenericDatumWriter, overriding writeArray()
>
> or s/Generic/Specific/ if you're using generated classes.
>
> Note that the reflect implementation already supports java primitive
> arrays, but values are boxed on read and write.  However if you write
> type-specific loops for int, float, long and double arrays you should be
> able to avoid any boxing.
>
> Please tell me how this works for you.
>
> Thanks,
>
> Doug
>

this is fantastic, thanks for the pointers doug.  i'm currently working on
some other stuff, but i'll let you know how this works out when i get back
to it.

PAt



Mime
View raw message