hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lakshmi Ramakrishnan (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HIVE-12299) Hive Column Data Type definition in schema limited to 4000 characters - too small
Date Thu, 29 Oct 2015 23:40:27 GMT
Lakshmi Ramakrishnan created HIVE-12299:
-------------------------------------------

             Summary: Hive Column Data Type definition in schema limited to 4000 characters
- too small
                 Key: HIVE-12299
                 URL: https://issues.apache.org/jira/browse/HIVE-12299
             Project: Hive
          Issue Type: Improvement
          Components: Hive
            Reporter: Lakshmi Ramakrishnan


The data type definitions in the table schema are limited to 4K characters - as per the code
here: https://github.com/apache/hive/tree/master/metastore/scripts/upgrade/derby

I checked quickly and all meta stores have similar schema definition for the column type.


Is there any reason why this limit is low? We had a table that had defined a struct, which
had over 200 columns and the column names were rather verbose (for readability). This caused
a non-obvious failure like 
FAILED: IllegalArgumentException Error: : expected at the end of 'string:array<struct<xyz:string,
.....'. Is it possible to make this configurable or at least increase it to something much
higher?

Additionally, there is no validation error that communicates this limitation to the user,
it required non-trivial debugging and looking into the table definitions when it failed trying
to parse what was essentially a truncated type. 





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message