hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lefty Leverenz (JIRA)" <>
Subject [jira] [Commented] (HIVE-9500) Support nested structs over 24 levels.
Date Wed, 04 Feb 2015 01:15:36 GMT


Lefty Leverenz commented on HIVE-9500:

Thanks for the javadoc changes, [~aihuaxu].  Now that you've changed the semicolon, I realize
the description was better the original way (a three-part sentence giving each possible case)
so if you're making other changes you might want to revert to the semicolon ... but then also
make "If" lowercase in the next clause.

Here's a new version with a few other edits:

+   * To be backward-compatible, initialize the first 3 separators to 
+   * be the given values. The default number of separators is 8; if only
+   * hive.serialization.extend.nesting.levels is set, extend the number of
+   * separators to 24; if hive.serialization.extend.additional.nesting.levels
+   * is set, extend the number of separators to 154.

By the way, in the first sentence what does "the given values" mean?

Wait ... I just noticed that the doc comments are on a private method.  So there won't be
any javadocs.

> Support nested structs over 24 levels.
> --------------------------------------
>                 Key: HIVE-9500
>                 URL:
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: Aihua Xu
>            Assignee: Aihua Xu
>              Labels: SerDe
>             Fix For: 1.2.0
>         Attachments: HIVE-9500.1.patch, HIVE-9500.2.patch
> Customer has deeply nested avro structure and is receiving the following error when performing
> 15/01/09 20:59:29 ERROR ql.Driver: FAILED: SemanticException org.apache.hadoop.hive.serde2.SerDeException:
Number of levels of nesting supported for LazySimpleSerde is 23 Unable to work with level
> Currently we support up to 24 levels of nested structs when hive.serialization.extend.nesting.levels
is set to true, while the customers have the requirement to support more than that. 
> It would be better to make the supported levels configurable or completely removed (i.e.,
we can support any number of levels). 

This message was sent by Atlassian JIRA

View raw message