hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thejas M Nair (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-9500) Support nested structs over 24 levels.
Date Tue, 10 Feb 2015 20:31:12 GMT

     [ https://issues.apache.org/jira/browse/HIVE-9500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Thejas M Nair updated HIVE-9500:
--------------------------------
    Attachment: TClass.java

If the theory is not convincing, attaching TClass.java that compares the performance.

The results -
{noformat}
$ java  -server  testpkg.TClass
Array escape time(ms) 5919
Num chars escaped 200000000
Map escape time(ms) 23275
Num chars escaped 200000000
Perf diff factor = 3.9322520696063523
{noformat}

> Support nested structs over 24 levels.
> --------------------------------------
>
>                 Key: HIVE-9500
>                 URL: https://issues.apache.org/jira/browse/HIVE-9500
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: Aihua Xu
>            Assignee: Aihua Xu
>              Labels: SerDe
>             Fix For: 1.2.0
>
>         Attachments: HIVE-9500.1.patch, HIVE-9500.2.patch, HIVE-9500.3.patch, TClass.java
>
>
> Customer has deeply nested avro structure and is receiving the following error when performing
queries.
> 15/01/09 20:59:29 ERROR ql.Driver: FAILED: SemanticException org.apache.hadoop.hive.serde2.SerDeException:
Number of levels of nesting supported for LazySimpleSerde is 23 Unable to work with level
24
> Currently we support up to 24 levels of nested structs when hive.serialization.extend.nesting.levels
is set to true, while the customers have the requirement to support more than that. 
> It would be better to make the supported levels configurable or completely removed (i.e.,
we can support any number of levels). 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message