avro-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Erik Frey (JIRA)" <j...@apache.org>
Subject [jira] Updated: (AVRO-673) Reduce time spent validating schemas
Date Fri, 24 Sep 2010 21:28:33 GMT

     [ https://issues.apache.org/jira/browse/AVRO-673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Erik Frey updated AVRO-673:
---------------------------

    Attachment: AVRO-673.patch

Ensures validation is done only once in the .write() method.  In an adhoc test, this reduced
the time to serialize a datafile with a complex schema from 8 seconds to 5.5 seconds.  Also
includes a small test to ensure AvroTypeException is thrown before and after the patch.

> Reduce time spent validating schemas
> ------------------------------------
>
>                 Key: AVRO-673
>                 URL: https://issues.apache.org/jira/browse/AVRO-673
>             Project: Avro
>          Issue Type: Improvement
>          Components: python
>            Reporter: Erik Frey
>            Priority: Minor
>         Attachments: AVRO-673.patch
>
>
> avro.io has a validate method that currently occupies around half the time it takes to
serialize a fairly complex record through a datafile.  validate() gets called repeatedly during
an object's traversal, even though validate itself is already recursive.  This introduces
combinatorially excessive validation that has a significant impact on the performance of serializing
complex records.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message