hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lefty Leverenz (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-9815) Metastore column"SERDE_PARAMS"."PARAM_VALUE" limited to 4000 bytes
Date Tue, 28 Mar 2017 00:21:41 GMT

    [ https://issues.apache.org/jira/browse/HIVE-9815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15944287#comment-15944287
] 

Lefty Leverenz commented on HIVE-9815:
--------------------------------------

Is any wiki documentation needed?

Repeating [~sambit19]'s question from January 21:

{quote}
Should this be updated under https://cwiki.apache.org/confluence/display/Hive/AdminManual+MetastoreAdmin#AdminManualMetastoreAdmin-SupportedBackendDatabasesforMetastore?


1. If the user doing a raw Oracle database installation for meta store, then  MAX_STRING_SIZE
setting is to be changed from STANDARD to EXTENDED.

2. If the user is planning to use an existing database, a warning should describe this situation
and suggest to use some other database.
{quote}

> Metastore column"SERDE_PARAMS"."PARAM_VALUE"  limited to 4000 bytes
> -------------------------------------------------------------------
>
>                 Key: HIVE-9815
>                 URL: https://issues.apache.org/jira/browse/HIVE-9815
>             Project: Hive
>          Issue Type: Bug
>          Components: Metastore
>    Affects Versions: 0.14.0
>            Reporter: Naveen Gangam
>            Assignee: Naveen Gangam
>            Priority: Critical
>             Fix For: 3.0.0, 2.3.0
>
>         Attachments: Hv2.txt
>
>
> Currently, in the hive metastore schema, the length of the column SERDE_PARAMS.PARAM_VALUE
is set to 4000 bytes. This is not enough for users that have a key with a value larger than
4000 bytes. Say something like hbase.columns.mapping.
> I am not a database historian but appears that this limitation may have been put in place
because Oracle's varchar2 was restricted to 4k bytes for a long time until recently. 
> According to the following documentation, even today Oracle DB's varchar2 only supports
a max size of 4000 unless a configuration parameter MAX_STRING_SIZE is set to EXTENDED.
> http://docs.oracle.com/database/121/SQLRF/sql_elements001.htm#SQLRF55623
> {code}
> MAX_STRING_SIZE=EXTENDED
> {code}
> Postgres supports a max of 1GB for character datatype according to http://www.postgresql.org/docs/8.3/static/datatype-character.html
> MySQL can support upto 65535 bytes for the entire row. So long as the PARAM_KEY value
+ PARAM_VALUE is less than 65535, we should be good.
> http://dev.mysql.com/doc/refman/5.0/en/char.html
> SQL Server's varchar max length is 8000 and can go beyond using "varchar(max)".
> http://dev.mysql.com/doc/refman/5.0/en/char.html
> Derby's varchar can be upto 32672 bytes.
> https://db.apache.org/derby/docs/10.7/ref/rrefsqlj41207.html



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message