hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mark Wagner (JIRA)" <>
Subject [jira] [Updated] (HIVE-10187) Avro backed tables don't handle cyclical or recursive records
Date Wed, 01 Apr 2015 22:28:53 GMT


Mark Wagner updated HIVE-10187:
    Attachment: HIVE-10187.demo.patch

Here's a patch which extends the testcase added in HIVE-7653 to illustrate the issue.

The problem here is that the generated TypeInfo for these data sets replaces the recursive
fields with void, so any serializer will faithfully serialize nulls instead of any relevant
data. However, making the TypeInfo properly recursive is likely to not work with other components
of Hive. At the very least, the 'describe' statement will cause an infinite loop. I believe
these issues could be fixed, but it would require making cyclical/recursive datasets a first
class citizen in Hive's data model. 

> Avro backed tables don't handle cyclical or recursive records
> -------------------------------------------------------------
>                 Key: HIVE-10187
>                 URL:
>             Project: Hive
>          Issue Type: Bug
>          Components: Serializers/Deserializers
>    Affects Versions: 1.2.0
>            Reporter: Mark Wagner
>         Attachments: HIVE-10187.demo.patch
> [HIVE-7653] changed the Avro SerDe to make it generate TypeInfos even for recursive/cyclical
schemas. However, any attempt to serialize data which exploits that ability results in silently
dropped fields.

This message was sent by Atlassian JIRA

View raw message