avro-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Doug Cutting <cutt...@apache.org>
Subject Re: Promote to string
Date Fri, 04 Feb 2011 18:49:39 GMT
On 02/04/2011 10:02 AM, Scott Carey wrote:
> In Avro Java API language, we have a ResolvingDecoder that handles all the
> basic schema reader/writer evolution and promotion.  A new
> 'TransformingDecoder' could be supplied with more advanced type
> transformation options.  Each type of transformation would need to be well
> defined.  If it was a general Avro tool and not only Java, it would
> require additions to the spec.

I think it makes sense to implement something like this first (e.g., in 
Java) to working out the details before adding it to the spec.  Ideally 
we might implement it in two languages before adding it to the spec if 
it seems like details might be language-dependent.

We should probably better define features in the spec so that we can 
better describe implementations.  I think the distinct specification 
features that an implementation might implement are roughly:

  - binary-format i/o of data corresponding to a schema
  - json-format i/o
  - sort-order comparison
  - aliases
  - container file format
  - rpc over http

I'd hope to soon add RPC over sockets using SASL to the specification, 
but am waiting for another implementation or two.

All of these are optional, though most depend on binary i/o.  I don't 
think that, e.g., schema resolution, default values and the deflate 
codec are optional, even though they may not be implemented in every 
implementation.  When they're not implemented I think that's a bug in 
the implementation of a specification feature.

Do folks agree with this categorization?

The following table should probably better correspond to this:



View raw message