avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bernardo Bennett <bernardo.benn...@gmail.com>
Subject String Pooling on reader side
Date Thu, 31 Mar 2016 16:40:50 GMT
Are there plans to introduce such feature? Depending on the nature of the
data, memory savings can be quite substantial.

So far I've experimented modifying the java generated IndexedRecord.put()
methods to perform lookups on concurrent hash maps in case field type is
String. The overhead seems insignificant compared to savings on GC times
and disk spills (Spark) for applications which read and cache avros in
memory.

Mime
View raw message