avro-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anders Sundelin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AVRO-1931) SchemaCompatibility fails to recognize reader compatible with all branches of a union
Date Tue, 11 Oct 2016 12:39:21 GMT

    [ https://issues.apache.org/jira/browse/AVRO-1931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15565311#comment-15565311

Anders Sundelin commented on AVRO-1931:

After investigating this a little further, I discovered that float unions are actually not
capable of reading int or long unions (though simple floats are). See attached patch, where
more test cases are added to the prior fix for this issue. 
I do not know if this is a real problem or not, but as the float promotion of ints and longs
already is lossy (mentioned in other comments), maybe we should not allow float unions to
read int or long unions? But, on the other hand, regular floats can do it?
I will not work more on this issue for now, as we do not use floats in our app. Instead, I
will focus on the error reporting of the SchemaCompatibility class. Feel free to investigate
or comment further on this. (The patch still applies, of course, and is applicable for all
other unions)

> SchemaCompatibility fails to recognize reader compatible with all branches of a union
> -------------------------------------------------------------------------------------
>                 Key: AVRO-1931
>                 URL: https://issues.apache.org/jira/browse/AVRO-1931
>             Project: Avro
>          Issue Type: Bug
>          Components: java
>    Affects Versions: 1.8.1
>         Environment: Java
>            Reporter: Anders Sundelin
>            Priority: Minor
>         Attachments: AVRO-1931-2.patch
>   Original Estimate: 1h
>  Remaining Estimate: 1h
> It is stated in the Avro spec
> "if writer's is a union, but reader's is not:
> If the reader's schema matches the selected writer's schema, it is recursively resolved
against it. If they do not match, an error is signalled."
> In case a the chosen reader is compatible with all branches of the union in the writer,
then the class SchemaCompatibility should reflect this. Currently it does not.
> The submitted patch corrects this (also added tests showing this behaviour in Avro)
> The new tests, in the class TestReadingWritingDataInEvolvedSchemas, could be redundant,
but they were very useful when exploring how Avro actually works during de-/serialization
> I will try to continue working a little bit on the SchemaCompatibility class, adding
more user-friendly error messages (suitable for deeper structures than todays error message).
Feel free to contact me if you have any ideas or pointers to existing work.

This message was sent by Atlassian JIRA

View raw message