hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Siying Dong (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HIVE-1750) Remove Partition Filtering Conditions when Possible
Date Wed, 27 Oct 2010 01:24:20 GMT

    [ https://issues.apache.org/jira/browse/HIVE-1750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12925230#action_12925230
] 

Siying Dong commented on HIVE-1750:
-----------------------------------

How about doing this (something is much more expensive but should be right):

We go throught the whole expression tree, for every node, we keep a vector of results. Each
result is for one partition, being true, false or null.
When doing logical expression, we do logical expression for every vector. Every for any node,
all the result of the element is all true or all false, we can replace it with the constant
true or false, and potentially remove its parent logical operator.

Since we only replace nodes when we know the results for sure, this algorithm will guarantee
to be correct.

> Remove Partition Filtering Conditions when Possible
> ---------------------------------------------------
>
>                 Key: HIVE-1750
>                 URL: https://issues.apache.org/jira/browse/HIVE-1750
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: Siying Dong
>            Assignee: Siying Dong
>
> For some simple queries, partition filtering constraints take 8% of CPU time (now 16%
since we filter twice) even if the result is always true. When possible, we should remove
these constraints to save CPU times.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message