hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Shelukhin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-10122) Hive metastore filter-by-expression is broken for non-partition expressions
Date Fri, 27 Mar 2015 22:29:53 GMT

    [ https://issues.apache.org/jira/browse/HIVE-10122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14384808#comment-14384808
] 

Sergey Shelukhin commented on HIVE-10122:
-----------------------------------------

[~gopalv] [~mmokhtar] [~hagleitn] this could be a considerable perf regression starting from
~Hive 0.14 for metastore partition pruning.
It would seemingly (I am going based on what Thejas found for one query, haven't looked into
it) affect every query combining partition + non-partition filter, causing pruning to go thru
partition-name-based path on the client.
We need to fix this... I can look into it next week

> Hive metastore filter-by-expression is broken for non-partition expressions
> ---------------------------------------------------------------------------
>
>                 Key: HIVE-10122
>                 URL: https://issues.apache.org/jira/browse/HIVE-10122
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Sergey Shelukhin
>
> See https://issues.apache.org/jira/browse/HIVE-10091?focusedCommentId=14382413&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14382413
> These two lines of code
> {noformat}
>     // Replace virtual columns with nulls. See javadoc for details.
>     prunerExpr = removeNonPartCols(prunerExpr, extractPartColNames(tab), partColsUsedInFilter);
>     // Remove all parts that are not partition columns. See javadoc for details.
>     ExprNodeDesc compactExpr = compactExpr(prunerExpr.clone());
> {noformat}
> are supposed to take care of this; I see there were bunch of changes to this code over
some time, and now it appears to be broken.
> Thanks to [~thejas] for info.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message