avro-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Leo Romanoff (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AVRO-1282) Make use of the sun.misc.Unsafe class during serialization if a JDK supports it
Date Sat, 27 Apr 2013 05:44:15 GMT

    [ https://issues.apache.org/jira/browse/AVRO-1282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13643541#comment-13643541
] 

Leo Romanoff commented on AVRO-1282:
------------------------------------

Scott, this is great! Question: If you index indexes against Schemas and the same schema is
used for two different classes with different structure in memory, what would happen?


Regarding IndexedRecords: Have a look at my latest patch and take the class GenericRecordAccessor
for it (the name was not quite reflecting what it does. In principle it can work also with
IndexedRecords). It does almost what you need. The only differences is that it uses a name
instead of a position and GenericRecord. So, you can easily change it to take position as
constructor argument and use it for access. And you change it to use IndexedRecord. That's
it.
                
> Make use of the sun.misc.Unsafe class during serialization if a JDK supports it
> -------------------------------------------------------------------------------
>
>                 Key: AVRO-1282
>                 URL: https://issues.apache.org/jira/browse/AVRO-1282
>             Project: Avro
>          Issue Type: Improvement
>          Components: java
>    Affects Versions: 1.7.4
>            Reporter: Leo Romanoff
>            Priority: Minor
>         Attachments: AVRO-1282-s1.patch, AVRO-1282-s2.patch, AVRO-1282-s3.patch, avro-1282-v1.patch,
avro-1282-v2.patch, avro-1282-v3.patch, avro-1282-v4.patch, avro-1282-v5.patch, avro-1282-v6.patch,
avro-1282-v7.patch, avro-1282-v8.patch, TestUnsafeUtil.java
>
>
> Unsafe can be used to significantly speed up serialization process, if a JDK implementation
supports java.misc.Unsafe properly. Most JDKs running on PCs support it. Some platforms like
Android lack a proper support for Unsafe yet.
> There are two possibilities to use Unsafe for serialization:
> 1) Very quick access to the fields of objects. It is way faster than with the reflection-based
approach using Field.get/set
> 2) Input and Output streams can be using Unsafe to perform very quick input/output.
>  
> 3) More over, Unsafe makes it possible to serialize to/deserialize from off-heap memory
directly and very quickly, without any intermediate buffers allocated on heap. There is virtually
no overhead compared to the usual byte arrays.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message