hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Szehon Ho (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-8784) Querying partition does not work with JDO enabled against PostgreSQL
Date Sat, 22 Nov 2014 02:14:34 GMT

    [ https://issues.apache.org/jira/browse/HIVE-8784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221749#comment-14221749
] 

Szehon Ho commented on HIVE-8784:
---------------------------------

Sure, I will do that.

> Querying partition does not work with JDO enabled against PostgreSQL
> --------------------------------------------------------------------
>
>                 Key: HIVE-8784
>                 URL: https://issues.apache.org/jira/browse/HIVE-8784
>             Project: Hive
>          Issue Type: Bug
>          Components: Metastore
>    Affects Versions: 0.15.0
>            Reporter: Chaoyu Tang
>            Assignee: Chaoyu Tang
>             Fix For: 0.15.0
>
>         Attachments: HIVE-8784.1.patch, HIVE-8784.1.patch, HIVE-8784.patch, HIVE-8784_1.patch
>
>
> Querying a partition in PostgreSQL fails when using JDO (with hive.metastore.try.direct.sql=false)
. Following is the reproduce example:
> {code}
> create table partition_test_multilevel (key string, value string) partitioned by (level1
string, level2 string, level3 string);
> insert overwrite table partition_test_multilevel partition(level1='1111', level2='111',
level3='11') select key, value from srcpart tablesample (11 rows);
> insert overwrite table partition_test_multilevel partition(level1='1111', level2='222',
level3='11') select key, value from srcpart tablesample (15 rows);
> insert overwrite table partition_test_multilevel partition(level1='1111', level2='333',
level3='11') select key, value from srcpart tablesample (20 rows);
> select level1, level2, level3, count(*) from partition_test_multilevel where level2 <=
'222' group by level1, level2, level3;
> {code}
> The query fails with following error:
> {code}
>       Caused by: org.apache.hadoop.hive.ql.parse.SemanticException: MetaException(message:Invocation
of method "substring" on "StringExpression" requires argument 1 of type "NumericExpression")
> 	at org.apache.hadoop.hive.ql.optimizer.ppr.PartitionPruner.getPartitionsFromServer(PartitionPruner.java:392)
> 	at org.apache.hadoop.hive.ql.optimizer.ppr.PartitionPruner.prune(PartitionPruner.java:215)
> 	at org.apache.hadoop.hive.ql.optimizer.ppr.PartitionPruner.prune(PartitionPruner.java:139)
> 	at org.apache.hadoop.hive.ql.parse.ParseContext.getPrunedPartitions(ParseContext.java:619)
> 	at org.apache.hadoop.hive.ql.optimizer.pcr.PcrOpProcFactory$FilterPCR.process(PcrOpProcFactory.java:110)
> 	... 21 more
> {code}
> It is because the JDO pushdown filter generated for a query having inequality/between
partition predicate uses DN indexOf function which is not working properly with postgresql
(see http://www.datanucleus.org/servlet/jira/browse/NUCRDBMS-840) 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message