avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Patrick Linehan <pline...@plinehan.com>
Subject Re: java specific implementation uses GenericArray ?
Date Wed, 18 Aug 2010 22:37:34 GMT
On Wed, Aug 18, 2010 at 11:23 AM, Doug Cutting <cutting@apache.org> wrote:

> On 08/16/2010 02:46 PM, Patrick Linehan wrote:
>> does anyone have any suggestions for dealing with large lists/arrays of
>> primitive values in avro?
>> in my case (numerical algorithms), my naive mapping of a vector type
>> (mathematical vectors, not java Vectors) to an avro specific type
>> generates a GenericArray<Double>.  needless to say, i would prefer to
>> avoid the cost of boxing up all the individual floating point numbers.
>> is it possible to coerce avro into using raw java primitive arrays, e.g.
>> "double[]"?
> It should be possible to subclass things to effect this.  In particular:
>  - extend GenericData, overriding isArray()
>  - extend GenericDatumReader, overriding readArray()
>  - extend GenericDatumWriter, overriding writeArray()
> or s/Generic/Specific/ if you're using generated classes.
> Note that the reflect implementation already supports java primitive
> arrays, but values are boxed on read and write.  However if you write
> type-specific loops for int, float, long and double arrays you should be
> able to avoid any boxing.
> Please tell me how this works for you.
> Thanks,
> Doug

this is fantastic, thanks for the pointers doug.  i'm currently working on
some other stuff, but i'll let you know how this works out when i get back
to it.


View raw message