avro-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Douglas Kaminsky (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AVRO-911) remove object reuse from Java APIs
Date Thu, 06 Oct 2011 03:16:30 GMT

    [ https://issues.apache.org/jira/browse/AVRO-911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13121695#comment-13121695
] 

Douglas Kaminsky commented on AVRO-911:
---------------------------------------

Object reuse is not about a direct throughput/latency improvement, it's about a consistent
processing velocity. Unpredictable GC causes disruptions in processing speed, which is unacceptable
for real time systems (at least, from the perspective of my current project), where we are
measuring latency in sub-millisecond scale.

I would support refactoring the reuse behavior into a subclass of the existing reader, but
not eliminating it.
                
> remove object reuse from Java APIs
> ----------------------------------
>
>                 Key: AVRO-911
>                 URL: https://issues.apache.org/jira/browse/AVRO-911
>             Project: Avro
>          Issue Type: Improvement
>          Components: java
>            Reporter: Doug Cutting
>            Assignee: Doug Cutting
>             Fix For: 1.6.0
>
>         Attachments: perf-reuse.patch
>
>
> Avro's Java APIs were designed to permit object reuse when reading with the assumption
that would provide performance advantages.  In particular, the old parameter in DatumReader<T>.read(T
old, Decoder), the Utf8 class, and the GenericArray.peek() method were all designed for this
purpose.  But I am unable to see significant performance improvements when objects are reused.
 I tried modifying Perf.java's GenericTest to reuse records, and its StringTest to not reuse
Utf8 instances and, in both cases, performance is not substantially altered.
> If we were to remove these then issues such as AVRO-803 would disappear.  Always using
java.lang.String instead of Utf8 would remove a lot of user confusion. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message