hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Prasad Chakka (JIRA)" <>
Subject [jira] Commented: (HIVE-750) new partitionpruner does not work with test mode
Date Thu, 03 Sep 2009 17:59:57 GMT


Prasad Chakka commented on HIVE-750:

predicate pushdown was optimization before the new pruner came along so this is not a bug
but a lack of feature :)

patch looks good but since a new child filter operator is created every time there is a non-deterministic
predicate in an expression, there will be two identical filter operators if there are two
non-deterministic are encountered while walking down the tree. 

a better option would be

1) merge the children's pushdown preds with the current filter op's preds
2) run the ppd algorithm to extract pushdown predicates
3) if step 2 does not return an empty set then skip to step 6 (new)
4) if the current op's child is a filter operator then skip to step 6 (new)
5) create a new filter op with the child's pushdown preds (new)
6) pass the results of step2 to current op's parent

so the above algorithm would create a new filter-op whenever a predicate can't be pushed down.
this can help in case multiple sub-queries where some of the predicates can be pushed down
one join but not all the joins in the query.

> new partitionpruner does not work with test mode
> ------------------------------------------------
>                 Key: HIVE-750
>                 URL:
>             Project: Hadoop Hive
>          Issue Type: Bug
>          Components: Query Processor
>            Reporter: Namit Jain
>            Assignee: Namit Jain
>            Priority: Critical
>         Attachments: hive.750.1.patch
> set hive.test.mode=true;
> the new partition pruner does not work

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message