hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Elliot West (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-12274) Increase width of columns used for general configuration in the metastore.
Date Mon, 08 Feb 2016 14:50:40 GMT

    [ https://issues.apache.org/jira/browse/HIVE-12274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15137023#comment-15137023
] 

Elliot West commented on HIVE-12274:
------------------------------------

Note that although the schema given in the example is contrived, we do see real world schemas
that easily exceed the upper length bound. I'm not in a position to publicly share them however.

> Increase width of columns used for general configuration in the metastore.
> --------------------------------------------------------------------------
>
>                 Key: HIVE-12274
>                 URL: https://issues.apache.org/jira/browse/HIVE-12274
>             Project: Hive
>          Issue Type: Improvement
>          Components: Metastore
>    Affects Versions: 2.0.0
>            Reporter: Elliot West
>            Assignee: Sushanth Sowmyan
>              Labels: metastore
>         Attachments: HIVE-12274.example.ddl.hql
>
>
> This issue is very similar in principle to HIVE-1364. We are hitting a limit when processing
JSON data that has a large nested schema. The struct definition is truncated when inserted
into the metastore database column {{COLUMNS_V2.YPE_NAME}} as it is greater than 4000 characters
in length.
> Given that the purpose of these columns is to hold very loosely defined configuration
values it seems rather limiting to impose such a relatively low length bound. One can imagine
that valid use cases will arise where reasonable parameter/property values exceed the current
limit. Can these columns not use CLOB-like types as for example as used by {{TBLS.VIEW_EXPANDED_TEXT}}?
It would seem that suitable type equivalents exist for all targeted database platforms:
> * MySQL: {{mediumtext}}
> * Postgres: {{text}}
> * Oracle: {{CLOB}}
> * Derby: {{LONG VARCHAR}}
> I'd suggest that the candidates for type change are:
> * {{COLUMNS_V2.TYPE_NAME}}
> * {{TABLE_PARAMS.PARAM_VALUE}}
> * {{SERDE_PARAMS.PARAM_VALUE}}
> * {{SD_PARAMS.PARAM_VALUE}}
> Finally, will this limitation persist in the work resulting from HIVE-9452?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message