avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dan Burkert <danburk...@gmail.com>
Subject Re: Effort towards Avro 2.0?
Date Wed, 04 Dec 2013 01:28:13 GMT
It would be nice if specific records implemented the generic record API.
 This is useful when you don't know field names of a specific record at
compile time.  Reflection is the only way around this currently.  Also, it
would be useful if there was built in support for determining if a given
schema is reader-compatible with a writer-schema.  Neither of these should
require changing the data serialization format.

- Dan

On Tue, Dec 3, 2013 at 7:20 AM, Doug Cutting <cutting@apache.org> wrote:

> On Mon, Dec 2, 2013 at 1:56 PM, Philip Zeyliger <philip@cloudera.com>
> wrote:
> > It sounds like you're proposing to break language API compatibility.  Are
> > you also proposing to break wire compatibility for Avro HTTP RPC, Avro
> Netty
> > RPC, and/or Avro datafiles?
>
> We should be able to provide back-compatibility.  When current APIs
> cannot be back-compatibly extended, new APIs can be added.  Old APIs
> may be deprecated but should be retained for a while.  Data files
> written by 1.x should be readable by 2.x.
>
> Forward compatibility may not be possible when new schema features are
> used.  Data files written in 2.x may not be readable by 1.x.  Perhaps
> we could add a mode that forces 2.x to write a 1.x format file.
>
> RPC interoperability requires that 2.x be able to both read and write
> 1.x format.  So long as a 1.x protocol is used, then 1.x and 2.x
> clients and servers might be able to interoperate using 1.x wire
> formats.  But when 2.x schema features are used this may not be
> possible.
>
> Perhaps we should proceed by making back-compatibility (ability to
> read 1.x) a requirement, then adding interoperabilty features (ability
> to write 1.x) as needed?
>
> Should we require that all new schema features (named unions,
> extensions, date primitive, etc,) have a lossless translation to a 1.x
> schema?
>
> Doug
>

Mime
View raw message