hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Raghotham Murthy (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HIVE-578) Refactor partition pruning code as an optimizer transformation
Date Wed, 15 Jul 2009 16:20:14 GMT

    [ https://issues.apache.org/jira/browse/HIVE-578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12731540#action_12731540

Raghotham Murthy commented on HIVE-578:

I am trying to make similar changes to input pruning for sampling. 

A few questions:

How come we have both tableDesc and tableScanDesc? I see that tableDesc is used in FetchWork
and tableScanDesc in TableScanOperator. Does it make sense to make FetchWork use TableScanOperator
(FetchOperator is not really an operator)? Then, merge tableDesc and tableScanDesc? This will
also allow us to have the same code path for select * queries as well. Right now it looks
like FetchOperator is duplicating some code. Also, sample pruning can use the merged tableScanDesc
object to store the sampling information.

> Refactor partition pruning code as an optimizer transformation
> --------------------------------------------------------------
>                 Key: HIVE-578
>                 URL: https://issues.apache.org/jira/browse/HIVE-578
>             Project: Hadoop Hive
>          Issue Type: Bug
>          Components: Query Processor
>    Affects Versions: 0.3.0
>            Reporter: Ashish Thusoo
>            Assignee: Ashish Thusoo
>         Attachments: patch-578.txt
> Some bugs with partition pruning have been reported and the correct fix for many of them
is to rewrite the partition pruning code as an optimizer transformation which gets kicked
in after the predicate pushdown code. This refactor also uses the graph walker framework so
that the partition pruning code gets consolidated well with the frameworks and does not work
on the query block but rather works on the operator tree.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message