hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Remus Rusanu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-16113) PartitionPruner::removeNonPartCols needs to handle AND/OR cases
Date Tue, 09 May 2017 17:30:04 GMT

    [ https://issues.apache.org/jira/browse/HIVE-16113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003117#comment-16003117
] 

Remus Rusanu commented on HIVE-16113:
-------------------------------------

{{OR}}: the partition pruner is reducing the expression to {{partcol = 1}} and this reduces
the list of partitions retrieved from MD:
{code}
    ppList = getPartitionsFromServer(tab, (ExprNodeGenericFuncDesc)compactExpr, conf, alias,
partColsUsedInFilter, oldFilter.equals(compactExpr.getExprString()));
{code}
By now things are already broken, but later the reduced partition list causes the {{PcrExprProcFactory.GenericFuncExprProcessor.handleUdfOr}}
to see a definite TRUE instead of a correct DIVIDE (since the filter expression is only evaluated
against a reduced list of partitions) and thus cause the {{PartitionConditionRemover}} to
remove the entire Filter. 

> PartitionPruner::removeNonPartCols needs to handle AND/OR cases
> ---------------------------------------------------------------
>
>                 Key: HIVE-16113
>                 URL: https://issues.apache.org/jira/browse/HIVE-16113
>             Project: Hive
>          Issue Type: Bug
>          Components: Logical Optimizer
>    Affects Versions: 1.2.1, 2.1.1, 2.2.0
>            Reporter: Gopal V
>            Assignee: Remus Rusanu
>         Attachments: HIVE-16113.1.patch
>
>
> {code}
> create table daysales (customer int) partitioned by (dt string);
> insert into daysales partition(dt='2001-01-01') values(1);
> select * from daysales where nvl(dt='2001-01-01' and customer=1, false);
> 0 ROWS
> {code}
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/PartitionPruner.java#L384
> {code}
> 2017-03-05T12:37:47,153  WARN [6f053d71-6ad6-4ad0-833d-337f2d499c82 main] ppr.PartitionPruner:
The expr = NVL(((dt = '2001-01-01') and null),false)
> {code}
> Because {{true and null => null}}, this turns into {{NVL(null, false)}} 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message