avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Wei Yang <...@weizilla.com>
Subject Re: DataFileWriter/Reader vs Encoder/Decoders in Java
Date Wed, 13 Jul 2011 21:27:56 GMT
Ahhh ok. I thought that json encoding was a fully implemented standard which
can be used alternatively with binary. Since it's not, it makes sense as to
why there's no application level API for it.

This clears up a lot of confusion. Thanks for your help Doug!
On Jul 13, 2011 3:57 PM, "Doug Cutting" <cutting@apache.org> wrote:
> There is not currently a standard Avro json-encoded file format.
>
> Note that most Avro implementations don't support the JSON encoding, and
> we want file formats to be maximally interoperable, so we perhaps
> shouldn't encourage folks to use the JSON encoding for persistent data.
>
> I have in one instance used the following textual format for
> JSON-encoded Avro data:
> - first line of file is schema
> - one JSON object per following line in file, conforming to schema
>
> Perhaps we could standardize on something like this if there are
> applications for an Avro textual, line-based file format.
>
> Doug
>
> On 07/13/2011 01:27 PM, Wei Yang wrote:
>> I read somewhere that it's suppose to be easy to switch between binary
>> encoding and json encoding for application testing and debugging. I also
>> read that schemas were always added to the data output so the schema
>> file itself is not required on the reader side. But after playing around
>> with the code, I only figured out how save the schema with the data in
>> binary (DataFileWriter method) or without the schema in json (Encoder
>> method). That's when I became confused as to which method to use and if
>> its possible to save data in json with a schema.
>>

Mime
View raw message