avro-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Scott Banachowski (JIRA)" <j...@apache.org>
Subject [jira] Updated: (AVRO-556) Poor performance for Reader::readBytes can be easily improved
Date Sat, 05 Jun 2010 05:16:28 GMT

     [ https://issues.apache.org/jira/browse/AVRO-556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Scott Banachowski updated AVRO-556:

    Status: Patch Available  (was: Open)

> Poor performance for Reader::readBytes can be easily improved
> -------------------------------------------------------------
>                 Key: AVRO-556
>                 URL: https://issues.apache.org/jira/browse/AVRO-556
>             Project: Avro
>          Issue Type: Improvement
>          Components: c++
>    Affects Versions: 1.3.2
>         Environment: Linux
>            Reporter: Dave Wright
>         Attachments: AVRO-556.patch
> The default implementation of Reader::readBytes on 1.3.2 reads bytes into the result
vector one-byte-at-a-time. For large byte arrays (~500k or so), this is horrendously slow.

> The code can easily be changed to simply do:
> {noformat}
> void readBytes(std::vector<uint8_t> &val) {
>         int64_t size = readSize(); 
>        val.resize(size);
>        in_.readBytes(&val[0], size);
> }
> {noformat}
> ..which will copy all the bytes in a single call.
> (note: it appears this function has been changed in the trunk, but it still copies byte-by-byte,
so the optimization would still apply).
> In my testing of serializing/deserializing a message with a 500k byte field in it 1000
times, execution time dropped from from 30+sec to 0.2sec with this optimization.
> The same optimization can easily be applied to readFixed(uint8_t *val...) as well.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message