hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mohit Sabharwal (JIRA)" <>
Subject [jira] [Commented] (HIVE-8205) Using strings in group type fails in ParquetSerDe
Date Tue, 11 Nov 2014 04:27:34 GMT


Mohit Sabharwal commented on HIVE-8205:

This jira fixes a bug in Char and Varchar support in Parquet SerDe. 

Limitations section already mentions that these data types are supported starting 0.14.0,
so no additional documentation is needed.

Thanks, [~leftylev].

> Using strings in group type fails in ParquetSerDe
> -------------------------------------------------
>                 Key: HIVE-8205
>                 URL:
>             Project: Hive
>          Issue Type: Bug
>          Components: Serializers/Deserializers
>            Reporter: Mohit Sabharwal
>            Assignee: Mohit Sabharwal
>              Labels: parquet
>             Fix For: 0.14.0
>         Attachments: HIVE-8205.1.patch, HIVE-8205.1.patch, HIVE-8205.patch
> In HIVE-7735, schema info was plumbed to ETypeConverter to disambiguate between hive
Char, Varchar and String types, which are all represented as PrimitiveType "binary" and OriginalType
"utf8" in parquet.
> However, this does not work for parquet nested types (that map to hive Array, Map, etc.)
containing these values, because schema lookup for nested values was not implemented.  It's
also non-trivial to do that in the current parquet serde implementation. Instead of plumbing
in the schema, we should convert 
> these types to the same Text writeable and let the object inspectors handle the final

This message was sent by Atlassian JIRA

View raw message