hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jakob Homan (JIRA)" <>
Subject [jira] [Commented] (HIVE-2390) LazyBinarySerde (and others) don't support unions
Date Thu, 18 Aug 2011 00:57:27 GMT


Jakob Homan commented on HIVE-2390:

Part of the problem is that the term union has been overloaded.  In SQL it means the actual
set union of two compatible data types, whereas in Avro and programming languages it means
one value that can be at any one time an instance of two or different types.  Union was added
as a full-on, first-class type by its inclusion in ObjectInspector's Category enum.  Is there
any reason not to expand this use to be more along the line of programming language's take
on unions?  If so, it should be marked as not really being a first-class type.  If not, support
for unions in all the serdes, in the grammar and in the documentation should be provided.

I would lobby for expanding its support as it's an important type in Avro and we're quite
hobbled by the inability to manipulate unioned values. (Avro handles nullable values by unioning
them with their type T and null, but Haivvreo transparently converts these just to the type
and returns null where appropriate. The problem lies in actual unions of non-null types, which
are less frequent but still valid.)

> LazyBinarySerde (and others) don't support unions
> -------------------------------------------------
>                 Key: HIVE-2390
>                 URL:
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Jakob Homan
> When the union type was introduced, full support for it wasn't provided.  For instance,
when working with a union that gets passed to LazyBinarySerde: 
> {noformat}Caused by: java.lang.RuntimeException: Unrecognized type: UNION
> 	at org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe.serialize(
> 	at org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe.serializeStruct(
> 	at org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe.serialize(
> {noformat}

This message is automatically generated by JIRA.
For more information on JIRA, see:


View raw message