hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Elliot West (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HIVE-12274) Increase width of columns used for general configuration in the metastore.
Date Tue, 27 Oct 2015 16:27:27 GMT
Elliot West created HIVE-12274:
----------------------------------

             Summary: Increase width of columns used for general configuration in the metastore.
                 Key: HIVE-12274
                 URL: https://issues.apache.org/jira/browse/HIVE-12274
             Project: Hive
          Issue Type: Improvement
          Components: Metastore
    Affects Versions: 2.0.0
            Reporter: Elliot West
            Assignee: Sushanth Sowmyan


This issue is very similar in principle to HIVE-1364. We are hitting a limit when processing
JSON data that has a large nested schema. The struct definition is truncated when inserted
into the metastore database column {{COLUMNS_V2.YPE_NAME}} as it is greater than 4000 characters
in length.

Given that the purpose of these columns is to hold very loosely defined configuration values
it seems rather limiting to impose such a relatively low length bound. One can imagine that
valid use cases will arise where reasonable parameter/property values exceed the current limit.
Can these columns not use CLOB-like types as for example as used by {{TBLS.VIEW_EXPANDED_TEXT}}?
It would seem that suitable type equivalents exist for all targeted database platforms:

* MySQL: {{mediumtext}}
* Postgres: {{text}}
* Oracle: {{CLOB}}
* Derby: {{LONG VARCHAR}}

I'd suggest that the candidates for type change are:

* {{COLUMNS_V2.TYPE_NAME}}
* {{TABLE_PARAMS.PARAM_VALUE}}
* {{SERDE_PARAMS.PARAM_VALUE}}
* {{SD_PARAMS.PARAM_VALUE}}

Finally, will this limitation persist in the work resulting from HIVE-9452?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message