avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sean Busbey <bus...@cloudera.com>
Subject Re: Adding new field with default value to an Avro schema
Date Tue, 03 Feb 2015 18:13:05 GMT
On Tue, Feb 3, 2015 at 11:01 AM, Burak Emre <emrekabakci@gmail.com> wrote:

> @Sean thanks for the explanation.
> I have multiple writers but only one reader and the only schema migration
> operation is adding a new field so I thought that I may use the same schema
> for all dataset since the ordering will be same in all of them even though
> some may contain extra fields which is also defined in schema definition.
> Actually I wanted to avoid using an external database for sequential
> schema ids since it would make the system more complex than it should be in
> my case but it seems this is the only option for now.

An external database isn't strictly required. The only important bit is
that each schema have a unique immutable identifier. As Doug mentioned, you
could do this as an enum of schemas in your source code (so long as you
handled updates in reader-then-writer order). Similarly, you could do it by
relying on schema fingerprints and just loading avsc files out of shared


View raw message