Mailing-List: contact issues-help@hive.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hive.apache.org
Date: Tue, 16 May 2017 14:24:04 +0000 (UTC)
From: "Remus Rusanu (JIRA)" <jira@apache.org>
To: issues@hive.apache.org
Message-ID: <JIRA.13072020.1494856474000.221507.1494944644931@Atlassian.JIRA>
In-Reply-To: <JIRA.13072020.1494856474000@Atlassian.JIRA>
References: <JIRA.13072020.1494856474000@Atlassian.JIRA> <JIRA.13072020.1494856474868@jira-lw-us.apache.org>
Subject: [jira] [Commented] (HIVE-16667) PostgreSQL metastore handling of
 CLOB types for COLUMNS_V2.TYPE_NAME and other field is incorrect
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Tue, 16 May 2017 14:24:11 -0000


    [ https://issues.apache.org/jira/browse/HIVE-16667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16012469#comment-16012469 ] 

Remus Rusanu commented on HIVE-16667:
-------------------------------------

The problem will not reproduce on Hive tables that were present at metastore upgrade. It must be a (partitioned?) table, created after the upgrade.

> PostgreSQL metastore handling of CLOB types for COLUMNS_V2.TYPE_NAME and other field is incorrect
> -------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-16667
>                 URL: https://issues.apache.org/jira/browse/HIVE-16667
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Remus Rusanu
>            Assignee: Naveen Gangam
>
> The CLOB JDO type introduced with HIVE-12274 does not work correctly with PostgreSQL. The value is written out-of-band and the LOB handle is written,as an INT, into the table. SELECTs return the INT value, which should had been read via the {{lo_get}} PG built-in, and then cast into string.
> Furthermore, the behavior is different between fields upgraded from earlier metastore versions (they retain their string storage) vs. values inserted after the upgrade (inserted as LOB roots).
> Teh code in {{MetasoreDirectSql.getPartitionsFromPartitionIds/extractSqlClob}} expects the underlying JDO/Datanucleus to map the column to a {{Clob}} but that does not happen, the value is a Java String containing the int which is the LOB root saved by PG.
> This manifests at runtime with errors like:
> {code}
> hive> select * from srcpart;
> Failed with exception java.io.IOException:java.lang.IllegalArgumentException: Error: type expected at the position 0 of '24030:24031' but '24030' is found.
> {code}
> the 24030:24031 should be 'string:string'.
> repro:
> {code}
> CREATE TABLE srcpart (key STRING COMMENT 'default', value STRING COMMENT 'default') PARTITIONED BY (ds STRING, hr STRING) STORED AS TEXTFILE;
> LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/kv1.txt" OVERWRITE INTO TABLE srcpart PARTITION (ds="2008-04-09", hr="11");
> select * from srcpart;
> {code}
> I did not see the issue being hit by non-partitioned/textfile tables, but that is just the luck of the path taken by the code. Inspection of my PG metastore shows all the CLOB fields suffering from this issue.


--
This message was sent by Atlassian JIRA
(v6.3.15#6346)