hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sean Busbey (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-3538) Avro SerDe can't handle Nullable Enums
Date Fri, 05 Oct 2012 13:40:02 GMT

    [ https://issues.apache.org/jira/browse/HIVE-3538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13470316#comment-13470316
] 

Sean Busbey commented on HIVE-3538:
-----------------------------------

I think at least part of this is a bug in Avro. If you follow the flow of canDeserializeNullableEnums
(in the attached patch of tests), inside of the first call to verifyNullableType GenericData
properly validates the record. Despite this, the call to GenericDatumWriter.write inside of
AvroGenericRecordWritable explodes because it's unable to resolve a String against a [null,
ENUM]. Especially when the same sequence of validate followed by write works in the non-union
case for canDeserializeEnums, that looks like a problem in Avro.

I'll follow up there, depending on the fix this issue might be obviated.
                
> Avro SerDe can't handle Nullable Enums
> --------------------------------------
>
>                 Key: HIVE-3538
>                 URL: https://issues.apache.org/jira/browse/HIVE-3538
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Sean Busbey
>         Attachments: HIVE-3538.tests.txt
>
>
> If a field has a schema that unions NULL with an enum, Avro fails to resolve the union
because Avro SerDe doesn't restore "enumness".
> Since the enum datum is a String, avro internals check the union for a string schema,
which is not present.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message