hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vineet Garg (Jira)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-23145) get_partitions_with_specs fails if filter expression is not parsable
Date Wed, 08 Apr 2020 21:20:00 GMT

     [ https://issues.apache.org/jira/browse/HIVE-23145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Vineet Garg updated HIVE-23145:
-------------------------------
    Attachment: HIVE-23145.1.patch

> get_partitions_with_specs fails if filter expression is not parsable
> --------------------------------------------------------------------
>
>                 Key: HIVE-23145
>                 URL: https://issues.apache.org/jira/browse/HIVE-23145
>             Project: Hive
>          Issue Type: Sub-task
>          Components: Metastore
>    Affects Versions: 4.0.0
>            Reporter: Vineet Garg
>            Priority: Major
>         Attachments: HIVE-23145.1.patch
>
>
> Expression is not parsable in most of the cases. Current API *get_partitions_by_expr*
anticipates this and provide a fallback mechanism. This basically deserialize the provided
expression, fetches all partition names for the table, prune partition names using the expression
and then uses the names to fetch required partition data.
>  Note that this expect serialized expression instead of string.
> This need to be done for both Direct SQL and JDO path.
> e.g. Following error is thrown for tpcds query 55 which provide expression *<column>
IS NOT NULL filter*
> *ERROR*
> {code:java}
> MetaException(message:Error parsing partition filter; lexer error: null; exception NoViableAltException(13@[]))MetaException(message:Error
parsing partition filter; lexer error: null; exception NoViableAltException(13@[])) at org.apache.hadoop.hive.metastore.PartFilterExprUtil.getFilterParser(PartFilterExprUtil.java:154)
at org.apache.hadoop.hive.metastore.ObjectStore$15.initExpressionTree(ObjectStore.java:4339)
at org.apache.hadoop.hive.metastore.ObjectStore$15.canUseDirectSql(ObjectStore.java:4319)
at org.apache.hadoop.hive.metastore.ObjectStore$GetHelper.start(ObjectStore.java:4021) at
org.apache.hadoop.hive.metastore.ObjectStore$GetHelper.run(ObjectStore.java:3985) at org.apache.hadoop.hive.metastore.ObjectStore.getPartitionSpecsByFilterAndProjection(ObjectStore.java:4395)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at
java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:97)
at com.sun.proxy.$Proxy26.getPartitionSpecsByFilterAndProjection(Unknown Source) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_partitions_with_specs(HiveMetaStore.java:5356)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at
java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
at com.sun.proxy.$Proxy27.get_partitions_with_specs(Unknown Source) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_partitions_with_specs.getResult(ThriftHiveMetastore.java:21620)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_partitions_with_specs.getResult(ThriftHiveMetastore.java:21604)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge.java:643)
at org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge.java:638)
at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876) at
org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:638)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message