hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aihua Xu (JIRA)" <>
Subject [jira] [Commented] (HIVE-9500) Support nested structs over 24 levels.
Date Thu, 05 Feb 2015 15:06:35 GMT


Aihua Xu commented on HIVE-9500:

+ [~xuefuz]  I was trying to create codereview while it never ends (maybe because it contains
many files?).  What I did are
1. I refactored the SerDeParamaters into separate class since it's used by multiple SerDe.
2. Add a new configuration hive.serialization.extend.additional.nesting.levels to support
over 24 levels.

[~leftylev] Thanks for catching the issues on the comments. Although it's private function,
I updated the comments by following your suggestion. 

> Support nested structs over 24 levels.
> --------------------------------------
>                 Key: HIVE-9500
>                 URL:
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: Aihua Xu
>            Assignee: Aihua Xu
>              Labels: SerDe
>             Fix For: 1.2.0
>         Attachments: HIVE-9500.1.patch, HIVE-9500.2.patch
> Customer has deeply nested avro structure and is receiving the following error when performing
> 15/01/09 20:59:29 ERROR ql.Driver: FAILED: SemanticException org.apache.hadoop.hive.serde2.SerDeException:
Number of levels of nesting supported for LazySimpleSerde is 23 Unable to work with level
> Currently we support up to 24 levels of nested structs when hive.serialization.extend.nesting.levels
is set to true, while the customers have the requirement to support more than that. 
> It would be better to make the supported levels configurable or completely removed (i.e.,
we can support any number of levels). 

This message was sent by Atlassian JIRA

View raw message