hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexander Pivovarov (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-10687) AvroDeserializer fails to deserialize evolved union fields
Date Thu, 14 May 2015 00:58:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-10687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14543016#comment-14543016
] 

Alexander Pivovarov commented on HIVE-10687:
--------------------------------------------

It would be easier to review if you add RB link to the Jira

> AvroDeserializer fails to deserialize evolved union fields
> ----------------------------------------------------------
>
>                 Key: HIVE-10687
>                 URL: https://issues.apache.org/jira/browse/HIVE-10687
>             Project: Hive
>          Issue Type: Bug
>          Components: Serializers/Deserializers
>            Reporter: Swarnim Kulkarni
>            Assignee: Swarnim Kulkarni
>         Attachments: HIVE-10687.1.patch
>
>
> Consider the union field:
> {noformat}
> union {int, string}
> {noformat}
> and now this field evolves to
> {noformat}
> union {null, int, string}.
> {noformat}
> Running it through the avro schema compatibility check[1], they are actually compatible
which means that the latter could be used to deserialize the data written with former. However
the avro deserializer fails to do that. Mainly because of the way it reads the tags from the
reader schema and then reds the corresponding data from the writer schema. [2]
> [1] http://pastebin.cerner.corp/31078
> [2] https://github.com/cloudera/hive/blob/cdh5.4.0-release/serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroDeserializer.java#L354



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message