avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ellisdp <david.el...@diegesis.co.uk>
Subject Re: Getting access to binary encoded data (C++)
Date Sun, 05 Jan 2014 14:12:50 GMT
OK, I have done the following, based on an example I found on another forum.

/( vector<uint8_t> &bytes ...)

	avro::EncoderPtr encoder = avro::binaryEncoder();

	auto_ptr<avro::OutputStream> out= avro::memoryOutputStream();

	encoder->init(*out);

	avro::encode(*encoder, obj);

	out->flush();

	size_t len = out->byteCount();

	auto_ptr<avro::InputStream> in = avro::memoryInputStream(*out);
	avro::StreamReader* reader = new avro::StreamReader(*in);

	bytes.reserve(len);

	while (reader->hasMore()) {
		uint8_t c = reader->read();
		bytes.push_back(c);
	}/

This works, though seems a little long-winded. Is this OK or is there a
simpler way?

Also, I had an issue with the buffer size used by "memoryOutputStream". This
allocates a default buffer size of 4096 bytes (as documented), and I found
that the length returned by "byteCount", and the number of bytes returned
from the stream reader, were always 4096, even though the size of the object
actually encoded was much smaller (150 bytes).

For our application this is not acceptable since we only want to use the
number of bytes actually required to encode the object. For now I have
changed the code to:

/	auto_ptr<avro::OutputStream> oStream = avro::memoryOutputStream(1);/

so that the memory used by the output stream grows one byte at a time, so
uses no more than the size actually needed to encode the object. This works,
though again seems a little clumsy - is there another way to find out the
size of the actual binary encoded data, as opposed to the buffer size?

Thanks.



--
View this message in context: http://apache-avro.679487.n3.nabble.com/Getting-access-to-binary-encoded-data-C-tp4028890p4028952.html
Sent from the Avro - Users mailing list archive at Nabble.com.

Mime
View raw message