avro-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thiruvalluvan M. G. (JIRA)" <j...@apache.org>
Subject [jira] Updated: (AVRO-345) Optimization for ResolvingDecoder
Date Tue, 19 Jan 2010 14:32:54 GMT

     [ https://issues.apache.org/jira/browse/AVRO-345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Thiruvalluvan M. G. updated AVRO-345:

    Resolution: Fixed
        Status: Resolved  (was: Patch Available)

Committed revision 900790. Thanks Doug for reviewing it.

bq. We talked about re-writing GenericDatumReader to use ResolvingDecoder as an optimization.
Did that end up being faster? If so, have you yet filed an issue for that?

I didn't see measurable improvement with GenericDatumReader using ResolvingDecoder. I'll profile
and if I can get some result, I'll file a JIRA.

> Optimization for ResolvingDecoder
> ---------------------------------
>                 Key: AVRO-345
>                 URL: https://issues.apache.org/jira/browse/AVRO-345
>             Project: Avro
>          Issue Type: Improvement
>          Components: java
>            Reporter: Thiruvalluvan M. G.
>            Assignee: Thiruvalluvan M. G.
>         Attachments: AVRO-345-test.patch, AVRO-345.patch
> If the writer's and reader's schema are records, we allow the order of fields to be different
in them. The ResolvingDecoder returns the fields in the writer's order. This is to avoid buffering.
The current implementation uses FieldAdjust action on the parser stack. The number of such
action symbols is equal to the number of reader fields. This causes performance problem because
the number of calls to advance() is almost double compared to the number of calls in ValidatingDecoder.
> This patch replaces the FieldAdjustAction symbols with FieldOrderAction symbols. We have
FieldOrderActions one per record, which is expected to be much smaller than the number of
fields. Though it changes the API for ResolvingDecoder slightly, there is no impact because
we do not have any users yet.
> I see a 10-15% improvement in my computer with Perf -S.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message