avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From levi12 <dslev...@gmail.com>
Subject Is there a java implementation to serialize a large record that does not fit in-memory?
Date Tue, 09 Sep 2014 17:14:49 GMT
I'm pretty new to avro and am trying to evaluate its potential as a
"container" for hdfs file contents.
I do know about SequenceFiles but would like a language agnostic file format
if possible.

>From the java api it seems like all object serialization occurs in-memory,
which seems to bound potential record size to the amount of available
memory. Is there another java workaround or implementation that I'm not
seeing? Explanations about feasibility (or lack of) would be very helpful.


View this message in context: http://apache-avro.679487.n3.nabble.com/Is-there-a-java-implementation-to-serialize-a-large-record-that-does-not-fit-in-memory-tp4030943.html
Sent from the Avro - Users mailing list archive at Nabble.com.

View raw message