avro-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Scott Banachowski <sbana...@yahoo-inc.com>
Subject clarifications on file format
Date Thu, 01 Apr 2010 00:23:29 GMT

I'm looking at the spec for the container file, and have 2 questions:

The map of metadata key/value pairs begins with a long, then a number of
string-key/bytes-value pairs.  To be consistent with avro maps, should this
be followed by a long of 0?  The spec doesn't say explicitly, but if the
header is described by an avro schema I would suspect yes.

Are the longs that describe the file block varint longs?  Or 64-bit longs?
I assume avro varints.  But if so, if you ever wanted to expand the size of
block by writing more objects to it, you'd be in trouble because you'd
potentially be unable to fit the new size in the varint's location.

Also, I looked around the repo for some example container files, but didn't
see any.  Are there any examples checked in that we can use to examine their
layout and test our readers?


View raw message