avro-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Douglas Kaminsky (Updated) (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (AVRO-985) Remove byte-by-byte copying in RecordBuilderBase.defaultValue for non-complex types
Date Tue, 20 Dec 2011 21:51:30 GMT

     [ https://issues.apache.org/jira/browse/AVRO-985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Douglas Kaminsky updated AVRO-985:
----------------------------------

    Component/s: java
    
> Remove byte-by-byte copying in RecordBuilderBase.defaultValue for non-complex types
> -----------------------------------------------------------------------------------
>
>                 Key: AVRO-985
>                 URL: https://issues.apache.org/jira/browse/AVRO-985
>             Project: Avro
>          Issue Type: Improvement
>          Components: java
>    Affects Versions: 1.6.1
>            Reporter: Douglas Kaminsky
>
> In one section of RecordBuilderBase.defaultValue(Field) (quoted below) a bytewise copy
of the default object is created based on the JSON value provided. However, this is an extremely
inefficient operation and causes large slowdowns when building large object sets, including
latency spikes when the binary encoder flushes. 
> A simple workaround for a majority of cases would be to have a separate code path for
"primitives" (fixed, string, boolean, int, double, enum, float, bytes) that allows direct
creation rather than a full bytewise copy (and subsequent deep copy).
> *_RecordBuilderBase.java_*:
> {code}
>     // If not cached, get the default Java value by encoding the default JSON
>     // value and then decoding it:
>     if (defaultValue == null) {
>       ByteArrayOutputStream baos = new ByteArrayOutputStream();
>       encoder = EncoderFactory.get().binaryEncoder(baos, encoder);
>       ResolvingGrammarGenerator.encode(
>           encoder, field.schema(), defaultJsonValue);
>       encoder.flush();
>       decoder = DecoderFactory.get().binaryDecoder(
>           baos.toByteArray(), decoder);
>       defaultValue = new GenericDatumReader(
>           field.schema()).read(null, decoder);
>       defaultSchemaValues.putIfAbsent(field.pos(), defaultValue);
>     }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message