avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Scott Carey <sc...@richrelevance.com>
Subject Re: mixed schema avro data file?
Date Wed, 01 Jun 2011 23:08:51 GMT
Two options:

* DIfferent files per schema
* One schema that is a union of all schemas you want in the file

Which is best depends on your use case.

On 6/1/11 4:02 PM, "Yang" <teddyyyy123@gmail.com<mailto:teddyyyy123@gmail.com>>

our use case is that we have many different types of "events", with different schemas.

I was thinking to dump them into one file, for easier maintenance of the files. but then I
found that all the DataFileWriter,
JsonEncoder/Decoder require a schema to be present, so each file can have really only one
schema. of course I can create a
separate encoder/writer for each record I write. but then there would be no way to parse out
the file later. such a mixed schema file can be
useful only to humans at best.

so generally what is your experience in dealing with serializing objects of different types?
do you put them in different files?


View raw message