avro-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ryan Blue (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (AVRO-1590) AllowNull causes unexpected resolution failure
Date Tue, 23 Sep 2014 17:58:38 GMT

     [ https://issues.apache.org/jira/browse/AVRO-1590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Ryan Blue updated AVRO-1590:
    Attachment: AVRO-1590-1.patch

AVRO-1590: Improve resolution of records in unions.

This fixes schema resolution when a record is inside a union and has a
name change. A common situation to encounter this error is using a
hand-written schema and switching to a reflected schema with AllowNull
(this is the included test case).

Previously, when a schema changed to a union, then bestBranch method
would only match records if the name and namespace matched exactly or if
both records were not named. The latter case assumed that any unnamed
records matched. This commit updates the bestBranch resolution so that a
Symbol is generated for the potential record pair and checks whether the 
result is an error action (if there are unresolved fields) or whether
the record fields are error actions (if a field exists but always
results in an error). This fixes the initial problem and chooses the 
correct branch in the rare case of multiple unnamed records.

There are two significant behavior changes when resolving an added union:
1. The first record with a matching structure will be chosen instead of one with a matching
2. Unnamed records are matched by structure rather than choosing the first in the union

> AllowNull causes unexpected resolution failure
> ----------------------------------------------
>                 Key: AVRO-1590
>                 URL: https://issues.apache.org/jira/browse/AVRO-1590
>             Project: Avro
>          Issue Type: Bug
>    Affects Versions: 1.7.7
>            Reporter: Ryan Blue
>            Assignee: Ryan Blue
>         Attachments: AVRO-1590-1.patch
> If I try to read data serialized with a hand-written schema using a reflected schema
generated by AllowNull, even though all of the individual types can be read. Here is a failing
> {code:java}
>   public static class Point {
>     double x;
>     double y;
>   }
>   public static class Circle {
>     Point center;
>     double radius;
>   }
>   @Test
>   public void testAllowNull() throws Exception {
>     new SchemaValidatorBuilder()
>         .canBeReadStrategy()
>         .validateLatest().validate(
>         SchemaBuilder.record("Circle").fields()
>             .name("center").type().record("Point").fields()
>             .requiredDouble("x")
>             .requiredDouble("y")
>             .endRecord().noDefault()
>             .requiredDouble("radius")
>             .endRecord(),
>         Arrays.asList(ReflectData.AllowNull.get().getSchema(Circle.class)));
>   }
> {code}
> The problem is that the {{ResolvingGrammarGenerator}} can't resolve a record with a nullable
record unless the two have the same [full name|https://github.com/apache/avro/blob/trunk/lang/java/avro/src/main/java/org/apache/avro/io/parsing/ResolvingGrammarGenerator.java#L439].
> It is more work, but I think the check should be whether there is a readable record in
the union. Another alternative is to special case a nullable union and allow the record-level
resolution only in that case.

This message was sent by Atlassian JIRA

View raw message