avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tibor Benke <bt...@balabit.hu>
Subject Re: Effort towards Avro 2.0?
Date Thu, 05 Dec 2013 08:48:11 GMT
In the case of RPC, I think, not just Python and Java, but all supported 
languages, should be compatible with each other.


On 12/02/2013 10:56 PM, Philip Zeyliger wrote:
> It sounds like you're proposing to break language API compatibility. 
>  Are you also proposing to break wire compatibility for Avro HTTP RPC, 
> Avro Netty RPC, and/or Avro datafiles?
> I'd be appreciative of a mechanism by which systems that happen to use 
> Avro currently need not be forced to choose one version or another. 
>  (One approach to this is to use a different package name.)
> As for adding to your list, I'd like to see a code-generated API for 
> Python.  (We like to call these APIs "specific" but I find that 
> terminology confusing.)
> -- Philip
> On Mon, Dec 2, 2013 at 1:42 PM, Christophe Taton 
> <christophe.taton@gmail.com <mailto:christophe.taton@gmail.com>> wrote:
>     Hi all,
>     Avro, in its current form, exhibits a number of limitations that
>     are hard to work with or around, and hard to fix within the scope
>     of Avro 1.x : fixing these issues would introduce incompatible
>     changes that warrant a major version bump, ie. Avro 2.0. An Avro
>     2.0 branch would be an opportunity to address most issues that
>     appeared held back for compatibility purposes so far.
>     I would like to initiate an effort in this direction and I am
>     willing to do the necessary work to gather and organize
>     requirements, and draft a design document for what Avro 2.0 would
>     look like. For this reason, if you have opinions regarding an Avro
>     2.0 branch or regarding issues and features that could fit in Avro
>     2.0, please reply to this thread.
>     To bootstrap, below is a list I gathered over the last couple of
>     years from several discussions:
>       * Specification
>           o Improved support for unions (incompatible change with
>             named unions and union properties).
>           o New extension data type, similar to ProtocolBuffer
>             extensions (incompatible change).
>           o Clear separation between Avro schema (data format) and
>             specific API client concerns: for example, the way Avro
>             strings are exposed through the Java API should not
>             pollute the schema definition. Each particular Java client
>             should configure their own decoders with the way they want
>             Avro strings to be represented.
>           o Clarification of compatibility and type promotion (safe
>             lossless conversions vs. best-effort lossy conversions):
>             promoting int to float potentially loses precision, which
>             is not necessarily acceptable for all clients. Avro
>             decoders should let clients configure which mode they need.
>       * IDL
>           o Generalized IDL for Avro schemas.
>           o Support for recursive records.
>           o Meta-schema : IDL definition for a schema.
>       * Java API
>           o Truly immutable schema objects (no properties / hashcode
>             mutation after construction).
>           o Immutable records.
>           o Complete record builder API (current record builders do
>             not play well with nested records).
>           o Complete generic API (there currently is no GenericUnion
>             or GenericMap).
>           o Improved unions support : union values as java.lang.Object
>             are less than ideal; union values could expose the union
>             branch through an enum (nulls could be handled specifically).
>       * Python 3 support
>       * RPC
>           o SASL support
>           o Full Python/Java parity and interoperability.
>     Please, comment or extend this list. Provided enough interest,
>     I'll happily digest feedback and organize it into a document (most
>     likely a wiki page?).
>     Thanks,
>     Christophe

View raw message