avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Zoltan Farkas <zolyfar...@yahoo.com>
Subject Re: Avro schema properties contention on multithread read
Date Sat, 08 Jul 2017 18:30:59 GMT
The order of attributes in Json might matter as far as I can remember, so LinkedHashMap might
not be replaceable with a concurrenthashmap.
Plus concurrenthashmap is not exactly without concurrency overhead…
I wrote a util that creates a immutable schema https://github.com/zolyfarkas/spf4j/blob/master/spf4j-avro/src/main/java/org/spf4j/avro/schema/Schemas.java#L26
<https://github.com/zolyfarkas/spf4j/blob/master/spf4j-avro/src/main/java/org/spf4j/avro/schema/Schemas.java#L26>

But you would have to use it it conjunction with a unsynchronized avro implementation. (which
I do in my fork, and you can do as well).

I wonder if there is interest in merging this into the avro lib someday.

—Z


> On Jul 6, 2017, at 12:20 PM, fady@legsem.com wrote:
> 
> On 05.07.2017 21:53, Zoltan Farkas wrote:
> 
>> The synchronization in JsonProperties is curently inconsistent (see getObjectProps())
which makes current implementation @NotThreadSafe
>>  
>> I think it would be probably best to remove synchronization from those methods...
and add @NotThreadSafe to the class...
>> Utilities like Schemas.synchronizedSchema(...) and Schemas.unmodifiableSchema(...)
could be added to help with various use cases...
>>  
>>  
>> —Z
>>  
> Thank you for your reply. I like your Schemas.unmodifiableSchema(...) a lot.
> 
> While what you are describing would be ideal, a simpler solution might be to change the
LinkedHashMap that backs jsonProperties into something like a ConcurrentHashMap, avoiding
the need for synchronization.
> 
> This being said ConcurrentHashMap itself does not preserve insertion order, so its not
a mere replacement to LinkedHashMap.
> 


Mime
View raw message