hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Prasad Chakka (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HIVE-750) new partitionpruner does not work with test mode
Date Thu, 03 Sep 2009 17:59:57 GMT

    [ https://issues.apache.org/jira/browse/HIVE-750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12751110#action_12751110
] 

Prasad Chakka commented on HIVE-750:
------------------------------------

predicate pushdown was optimization before the new pruner came along so this is not a bug
but a lack of feature :)

patch looks good but since a new child filter operator is created every time there is a non-deterministic
predicate in an expression, there will be two identical filter operators if there are two
non-deterministic are encountered while walking down the tree. 

a better option would be

1) merge the children's pushdown preds with the current filter op's preds
2) run the ppd algorithm to extract pushdown predicates
3) if step 2 does not return an empty set then skip to step 6 (new)
4) if the current op's child is a filter operator then skip to step 6 (new)
5) create a new filter op with the child's pushdown preds (new)
6) pass the results of step2 to current op's parent

so the above algorithm would create a new filter-op whenever a predicate can't be pushed down.
this can help in case multiple sub-queries where some of the predicates can be pushed down
one join but not all the joins in the query.

> new partitionpruner does not work with test mode
> ------------------------------------------------
>
>                 Key: HIVE-750
>                 URL: https://issues.apache.org/jira/browse/HIVE-750
>             Project: Hadoop Hive
>          Issue Type: Bug
>          Components: Query Processor
>            Reporter: Namit Jain
>            Assignee: Namit Jain
>            Priority: Critical
>         Attachments: hive.750.1.patch
>
>
> set hive.test.mode=true;
> the new partition pruner does not work

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message