avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Zoltan Farkas <zolyfar...@yahoo.com>
Subject Re: Avro schema properties contention on multithread read
Date Wed, 05 Jul 2017 19:53:27 GMT
The synchronization in JsonProperties is curently inconsistent (see getObjectProps()) which
makes current implementation @NotThreadSafe

I think it would be probably best to remove synchronization from those methods… and add
@NotThreadSafe to the class…
Utilities like Schemas.synchronizedSchema(…) and Schemas.unmodifiableSchema(…) could be
added to help with various use cases…


—Z


> On Jun 29, 2017, at 2:21 AM, fady@legsem.com wrote:
> 
> Hello,
> 
> We are using Avro Schema properties and while running concurrent tests, we noticed a
lot of contentions on org.apache.avro.JsonProperties#getJsonProp.
> 
> In the attached screen shot, we have 4 concurrent threads all sharing the same avro schema
and reading from it simultaneously.
> 
> On this screen shot each red period is a contention between threads. Most of these contentions
are on getJsonProp.
> 
> This is due to getJsonProp being a synchronized method.
> 
> We have tried avro 1.7.7, 1.8.1 and 1.8.2. All have this problem (getJsonProp is deprecated
in 1.8 but the replacement method is also synchronized).
> 
> We can work around this by not sharing the avro schemas between threads (using ThreadLocal
for instance) but this is ugly.
> 
> It seems that avro schemas are mostly immutable, which is great for multithread read
access, but it turns out Properties within these schemas are mutable and, since they are stored
in a LinkedHashMap, synchronization is necessary.
> 
> Anyone having a similar issue?
> 
> Thank you<pastedImage.png>


Mime
View raw message