avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Philip Zeyliger <phi...@cloudera.com>
Subject Re: Python avro performance
Date Sat, 10 Jan 2015 00:38:34 GMT
If I recall correctly, the Python write implementation does a recursive
pass to check the data against the schema.  This is sometimes necessary to
choose which branch of a union to take when you're faced with typeless
dicts, but it's done more often than necessary in the python
implementation, and is very slow.

I think the right approach is to have a way for the user to tag the various
dicts to indicate which branch of a union it'll represent.

-- Philip


On Fri, Jan 9, 2015 at 4:01 PM, Doug Cutting <cutting@apache.org> wrote:

> On Fri, Jan 9, 2015 at 6:05 AM, Bruce Mitchener
> <bruce.mitchener@gmail.com> wrote:
> > Has anyone profiled the Python code or otherwise looked at the
> performance?
>
> Not that I've heard of.  Optimizing Avro Python would be a great project.
>
> Doug
>

Mime
View raw message