avro-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Scott Carey (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (AVRO-1282) Make use of the sun.misc.Unsafe class during serialization if a JDK supports it
Date Fri, 26 Apr 2013 05:36:16 GMT

     [ https://issues.apache.org/jira/browse/AVRO-1282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Scott Carey updated AVRO-1282:

    Attachment: AVRO-1282-s3.patch

This patch brings the performance back up a little, but it is still slightly below the prior
level due to using a Map<String, FieldAccessor> instead of FieldAccessor[].

There are a couple other simplifications.

The read side is particularly slow for both arrays and fields -- there is room for improvement
in how the array read machinery works.

I'm likely done with my main burst of activity on this ticket for now -- feel free to pick
up where I've left off and experiment more.

All tests are passing, but I think we should add some more tests to cover more corner cases
before committing any of these variations -- this stuff is a little dangerous.

Latly, My patch does not contain all of Leo's changes to Perf.java -- please submit another
patch for that; I can't seem to get the latest one uploaded to work.
> Make use of the sun.misc.Unsafe class during serialization if a JDK supports it
> -------------------------------------------------------------------------------
>                 Key: AVRO-1282
>                 URL: https://issues.apache.org/jira/browse/AVRO-1282
>             Project: Avro
>          Issue Type: Improvement
>          Components: java
>    Affects Versions: 1.7.4
>            Reporter: Leo Romanoff
>            Priority: Minor
>         Attachments: AVRO-1282-s1.patch, AVRO-1282-s2.patch, AVRO-1282-s3.patch, avro-1282-v1.patch,
avro-1282-v2.patch, avro-1282-v3.patch, avro-1282-v4.patch, avro-1282-v5.patch, avro-1282-v6.patch,
avro-1282-v7.patch, avro-1282-v8.patch, TestUnsafeUtil.java
> Unsafe can be used to significantly speed up serialization process, if a JDK implementation
supports java.misc.Unsafe properly. Most JDKs running on PCs support it. Some platforms like
Android lack a proper support for Unsafe yet.
> There are two possibilities to use Unsafe for serialization:
> 1) Very quick access to the fields of objects. It is way faster than with the reflection-based
approach using Field.get/set
> 2) Input and Output streams can be using Unsafe to perform very quick input/output.
> 3) More over, Unsafe makes it possible to serialize to/deserialize from off-heap memory
directly and very quickly, without any intermediate buffers allocated on heap. There is virtually
no overhead compared to the usual byte arrays.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message