hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Elliot West (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-1364) Increase the maximum length of various metastore fields, and remove TYPE_NAME from COLUMNS primary key
Date Tue, 27 Oct 2015 15:58:27 GMT

    [ https://issues.apache.org/jira/browse/HIVE-1364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14976608#comment-14976608
] 

Elliot West commented on HIVE-1364:
-----------------------------------

Echoing [~dumpster]'s comment, we too are hitting this limit (also with nested JSON) and are
having to alter these tables by hand. Given that the purpose of these columns is to hold very
loosely defined configuration values it seems rather limiting to impose such a relatively
low length bound. Can we not use CLOB-like types as for example as used by {{TBLS.VIEW_EXPANDED_TEXT}}?

I'd suggest that the candidates for type change are:

* {{COLUMNS_V2.TYPE_NAME}}
* {{TABLE_PARAMS.PARAM_VALUE}}
* {{SERDE_PARAMS.PARAM_VALUE}}
* {{SD_PARAMS.PARAM_VALUE}}

Finally, will this limitation persist in the work resulting from HIVE-9452?

> Increase the maximum length of various metastore fields, and remove TYPE_NAME from COLUMNS
primary key
> ------------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-1364
>                 URL: https://issues.apache.org/jira/browse/HIVE-1364
>             Project: Hive
>          Issue Type: Bug
>          Components: Metastore
>    Affects Versions: 0.5.0
>            Reporter: Carl Steinbach
>            Assignee: Carl Steinbach
>             Fix For: 0.6.0
>
>         Attachments: HIVE-1364.2.patch.txt, HIVE-1364.3.backport-060.patch.txt, HIVE-1364.3.patch.txt,
HIVE-1364.4.backport-060.patch.txt, HIVE-1364.4.patch.txt, HIVE-1364.patch
>
>
> The value component of a SERDEPROPERTIES key/value pair is currently limited
> to a maximum length of 767 characters. I believe that the motivation for limiting the
length to 
> 767 characters is that this value is the maximum allowed length of an index in
> a MySQL database running on the InnoDB engine: http://bugs.mysql.com/bug.php?id=13315
> * The Metastore OR mapping currently limits many fields (including SERDEPROPERTIES.PARAM_VALUE)
to a maximum length of 767 characters despite the fact that these fields are not indexed.
> * The maximum length of a VARCHAR value in MySQL 5.0.3 and later is 65,535.
> * We can expect many users to hit the 767 character limit on SERDEPROPERTIES.PARAM_VALUE
when using the hbase.columns.mapping serdeproperty to map a table that has many columns.
> I propose increasing the maximum allowed length of SERDEPROPERTIES.PARAM_VALUE to 8192.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message