ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrey Kornev <andrewkor...@hotmail.com>
Subject Deserialization of BinaryObject's byte arrays
Date Fri, 03 Nov 2017 20:07:16 GMT
Hello,

We store lots of byte arrays (serialized graph-like data structures) as fields of BinaryObject.
Later on, while reading such field, BinaryInputStream implementation creates an on-heap array
and copies the bytes from the BinaryObject's internal backing array to the new array.

While in general case it works just fine, in our case, a lot of CPU is spent on copying of
the bytes and significant amount garbage is generated as well.

I'd like Ignite Community to consider the following proposal: introduce support for serialization/deserialization
of the ByteBuffer class.  A BinaryInputStream implementation would special case ByteBuffer
fields same way it currently special cases various Java types (collections, dates, timestamps,
enums, etc.)

The application developer would simply call BinaryObjectBuilder.setField(...) passing in an
instance of ByteBuffer. During serialization, the ByteBuffer's bytes would simply be copied
to the backing array (similar to how regular byte arrays are serialized today) and in the
binary object's metadata the field marked as, say, GridBinaryMarshaller.BYTE_BUFFER.

During deserialization of the field, instead of allocating a new byte array and transferring
the bytes from the BinaryObject's backing array, BinaryInputStream would simply wrap the required
sub-array into a read-only instance of ByteBuffer using "public static ByteBuffer wrap(byte[]
array, int offset, int length)" and return the instance to the application.

This approach doesn't require any array allocation/data copying and is faster and significantly
reduces pressure on the garbage collector. The benefits are especially great when application
stores significant amount of its data as byte arrays.

Clearly, this approach equally applies to arrays of other primitive types.

A minor complication does arise when dealing with off-heap BinaryObjects, but nothing that
can't solved with a little bit of Java reflection wizardry (or by other means).

Thanks,
Andrey

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message