avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Doug Cutting <cutt...@apache.org>
Subject Re: mixing types using reflection
Date Mon, 17 Aug 2009 19:04:48 GMT
Eelco Hillenius wrote:
> There are two show stoppers for me right now, AVRO-93 and AVRO-95.
> Now, my question is about the latter one, which is about mixing
> multiple types in one data file using reflection. I submitted a unit
> test for it that shows the bug, but I'm wondering if the way I'm using
> the API is as it is intended.

No, it is not.  An Avro data file is expected to contain instances that 
all conform to a single schema.  If you have multiple classes that you'd 
like to store in a single file then you can use a union schema, e.g.:

List<Schema> records = new ArrayList<Schema>();
records.add(ReflectData.getSchema(MyRecord1.class));
records.add(ReflectData.getSchema(MyRecord2.class));
records.add(ReflectData.getSchema(MyRecord3.class));
Schema union = Schema.createUnion(records);
DataFileWriter<Object> writer
  = new DataFileWriter<Object>(union, fos, new ReflectDatumWriter(union);

I have not tested this code, but it should work.

It should also work to add new schemas to the list as you write the 
file, before adding records of each type.

Doug

Mime
View raw message