avro-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kenneth Knowles (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AVRO-1283) remove static caches from GenericData, SpecificData and ReflectData
Date Sun, 30 Apr 2017 15:43:04 GMT

    [ https://issues.apache.org/jira/browse/AVRO-1283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15990268#comment-15990268

Kenneth Knowles commented on AVRO-1283:

I wanted to note that these caches also caused some trouble with classloaders in FLINK-5633
and BEAM-1970. Maybe this is known but since I didn't see that mentioned here, I thought I
would add a comment.

> remove static caches from GenericData, SpecificData and ReflectData
> -------------------------------------------------------------------
>                 Key: AVRO-1283
>                 URL: https://issues.apache.org/jira/browse/AVRO-1283
>             Project: Avro
>          Issue Type: Improvement
>            Reporter: Doug Cutting
> Currently GenericData, ReflectData and SpecificData are often treated as singletons,
through an instance returned by a static get() method.  Because of this, the caches they have
must be thread-safe and are also memory leaks.
> Increasingly of late, new instances of these are allocated for use by a given DatumReader
or DatumWriter.  This issue explores whether we can:
>  - Change the static get() methods to always return new instances, so that each DatumReader
and DatumWriter has its own instance.  Since DatumReader and DatumWriter are not thread-safe,
the thread safety requirement could then be removed from caches, increasing their performance;
>  - Change caches to be non-static, to remove potential memory leaks.

This message was sent by Atlassian JIRA

View raw message