avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bill Graham <billgra...@gmail.com>
Subject Re: Combining schemas
Date Wed, 10 Aug 2011 00:53:32 GMT
I see that Doug's already got an IDL patch in place for AVRO-872, so I'll
take a look at that (tonight hopefully) and comment on it. Maybe these two
items are one in the same though...

Scott I like your approach, but I wonder if it could be simplified. Maybe we
add two new methods like this instead:

public Schema Schema.parse(File[] files); // (1)
public Schema Schema.parse(File[] files, Map<Name, Schema> context); // (2)

1 - This allows the caller to not have to make multiple calls and handle
state if they don't want to.
2 - This does the same, but also exposes the context, which would be handy
to easily grab a specific schema. Now that I say it I wonder if (1) would
ever be useful on it's own without having the context?


On Tue, Aug 9, 2011 at 2:20 PM, Scott Carey <scottcarey@apache.org> wrote:

> On 8/9/11 1:45 PM, "Bill Graham" <billgraham@gmail.com> wrote:
>
> Thanks Scott and Doug, see follow up below.
>
> On Tue, Aug 9, 2011 at 11:42 AM, Scott Carey <scottcarey@apache.org>wrote:
>
>> On 8/9/11 11:15 AM, "Bill Graham" <billgraham@gmail.com> wrote:
>>
>>
>>  Using the lower level Avro API you can parse the files yourself in an
>> order that will work.
>
>
> How exactly would the approach work where you parse files in
> reverse-dependency order work? This is something I'd like to explore and
> maybe contribute a helper for. I've tried a few combinations of this
> approach to no avail:
>
>         Schema schema1 = Schema.parse(new
> File("examples/java/avro/position.avsc"));
>         Schema schema2 = schema1.parse(new
> File("examples/java/avro/player.avsc"));
>
>
> I was mistaken.  The methods to do this are not public.
>
> The internals of the parser in Schema.java use a hidden inner class, Names.
>  Names is a LinkedHashMap<Name, Schema> .
>
> It should be relatively easy to add a signature to Schema.java along the
> lines:
>
> parse(File schema, Map<Name, Schema> context);
>
> This would:
>   for the given context:
>      validate it (each Name should match a Named schema with the same name)
>      place names into a Names instance and parse the schema with the Names
> context
>   on return from the internal parse, copy the contents of the Names object
> into the provided context
>
> Alternatively we could make the Names class public, clean it up and
> document it, and expose parse methods that take Names objects.
>
>
> Then you could do:
>
> Map<Name, Schema> context = new HashMap<Name, Schema>();
> Schema schema1 = Schema.parse(new
> File("examples/java/avro/position.avsc", context));
> Schema schema2 = schema1.parse(new
> File("examples/java/avro/player.avsc", context));
>
>
>
>

Mime
View raw message