avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Fengyun RAO <raofeng...@gmail.com>
Subject Re: Is it possible to write a magic byte in Avro file head?
Date Mon, 19 May 2014 03:31:19 GMT
Thanks, Doug. The Avro file metadata is exactly what we need!


2014-05-17 7:20 GMT+08:00 Doug Cutting <cutting@apache.org>:

> This incompatibly alters the Avro file format.  Could you perhaps
> instead add this into the Avro file's metadata?
>
> Doug
>
> On Thu, May 15, 2014 at 5:44 AM, Fengyun RAO <raofengyun@gmail.com> wrote:
> > I have a cache file using Avro serialization, and I want to add a magic
> byte
> > indicating cache version at the beginning of the file.
> > I find it's easy to serialize, but difficult to deserialize in C#.
> > First I open a filestream, read my magic byte, and then pass the stream
> to
> > the DataFileReader:
> >
> > var reader = DataFileReader<Dictionary<string,
> MyType>>.OpenReader(stream,
> > CACHE_SCHEMA)
> >
> > but it throws an AvroRuntimeException("Not an Avro data file")
> >
> > I look into the OpenReader() method:
> >
> >   // verify magic header
> >   byte[] magic = new byte[DataFileConstants.Magic.Length];
> >   inStream.Seek(0, SeekOrigin.Begin);
> >
> > It will always seek back to the beginning of the FileStream (which
> includes
> > my own byte), and thus throws an Exception.
> >
> > However, in java version, I could use DataFileStream which wouldn't seek
> > back and it works.
> >
> > Is there a way to make it work in C# version? I also wonder why there
> isn't
> > an equivalent "DataFileStream" class in C#.
>

Mime
View raw message