Mailing-List: contact dev-help@avro.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@avro.apache.org
Received-SPF: pass (athena.apache.org: domain of christophe.taton@gmail.com
 designates 209.85.215.171 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CALEq1Z8mMwmzqBvCaHDZBNHnjquVbys3Gd6taevkpTqtu9dAxg@mail.gmail.com>
References: 
 <CADDC+rdz6JDU6=88+7+FObJGwf1VqSK3mnEF9iYOsoRP_KXtHQ@mail.gmail.com>
 <CALEq1Z8mMwmzqBvCaHDZBNHnjquVbys3Gd6taevkpTqtu9dAxg@mail.gmail.com>
From: Christophe Taton <christophe.taton@gmail.com>
Date: Wed, 4 Dec 2013 10:24:06 -0800
Message-ID: 
 <CADDC+rf+zX7JoRxbTe-8uxfBTViirFAeJeXBgXJdr4_TJHoqfw@mail.gmail.com>
Subject: Re: Effort towards Avro 2.0?
To: dev@avro.apache.org
Content-Type: multipart/alternative; boundary=001a11c3bc56e788a004ecb9851d

--001a11c3bc56e788a004ecb9851d
Content-Type: text/plain; charset=UTF-8

On Tue, Dec 3, 2013 at 7:49 AM, Doug Cutting <cutting@apache.org> wrote:

> On Mon, Dec 2, 2013 at 1:42 PM, Christophe Taton
> <christophe.taton@gmail.com> wrote:
> > - New extension data type, similar to ProtocolBuffer extensions
> (incompatible change).
>
> Extensions might be implemented as something like:
>
>   {"type":"record", "name":"extension", "fields":[
>     {"name":"fingerprint", "type": {"type":"fixed", "size":16}},
>     {"name":"payload", "type":"bytes"}
>     ]
>   }
>
> One could then use this with:
>
>   {"type":"record", "name":"Foo", "fields":[
>     {"name":"bar", "type":"extension"}
>     ]
>   }
>
> The implementation could then find the schema for the extension at
> runtime given its fingerprint.  The reader could have a table mapping
> fingerprints to schemas.
>
> In particular, the specific compiler, when it sees a schema like:
>
>
>   {"type":"record", "name":"Bar", "isExtension":true, "fields":[
>     {"name":"x", "type":"long"}
>     ]
>   }
>
> Might emit code to add entries to the extension mapping table used by
> SpecificDatumReader, e.g.:
>
>   static {
>     SpecificData.addExtension(getSchema());
>   }
>
> Might something like this work?
>

Yes, this is very much the idea.
In a prototype I made a few months ago, I found allowing the user to
specify the fingerprint schema useful : in some scenario, an extension
could be prefixed by a string that contains the JSON schema; in some other
scenario, I may want to use fingerprints to identify the schema of the
extension; in some other cases, I may want to use some external mapping
maintained by another system (eg. the schema repository worked on in
AVRO-1124).

C.

--001a11c3bc56e788a004ecb9851d--