hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Namit Jain (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HIVE-578) Refactor partition pruning code as an optimizer transformation
Date Thu, 23 Jul 2009 21:39:14 GMT

    [ https://issues.apache.org/jira/browse/HIVE-578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12734800#action_12734800
] 

Namit Jain commented on HIVE-578:
---------------------------------

Some more comments:

1. Add lots of new tests
2. Wrong comments in OpProcFactory:

      // If fop2 exists (i.e this is not the top level filter and fop2 is not
      // a sampling filter t1ehen we ignore the current filter
      if (fop2 != null && !fop2.getConf().getIsSamplingPred())
        return null;
      
      // ignore the predicate in case it is not a sampling predicate
      if (fop.getConf().getIsSamplingPred()) {
        return null;
      }



should be:


      // If fop2 exists (i.e this is not the top level filter and fop2 is not
      // a sampling filter then we ignore the current filter
      if (fop2 != null && !fop2.getConf().getIsSamplingPred())
        return null;
      
      // ignore the predicate in case it is a sampling predicate
      if (fop.getConf().getIsSamplingPred()) {
        return null;
      }


3. ExprProcFactory.java:

wrong parameters doc:
  /**
   * Extracts pushdown predicates from the given list of predicate expression
   * @param opContext operator context used for resolving column references
   * @param op operator of the predicates being processed
   * @param preds
   * @return hasNonPartCols returns true/false depending upon whether this pred has a non
partition column
   * @throws SemanticException
   */

4. ExprProcFactory: 104/146/182
shouldnt you check for exprNodeNullDesc also


5. wrong comment: 63
    /**
     * Converts the reference from child row resolver to current row resolver
     */


> Refactor partition pruning code as an optimizer transformation
> --------------------------------------------------------------
>
>                 Key: HIVE-578
>                 URL: https://issues.apache.org/jira/browse/HIVE-578
>             Project: Hadoop Hive
>          Issue Type: Bug
>          Components: Query Processor
>    Affects Versions: 0.3.0
>            Reporter: Ashish Thusoo
>            Assignee: Ashish Thusoo
>         Attachments: patch-578.txt, patch-578_1.txt
>
>
> Some bugs with partition pruning have been reported and the correct fix for many of them
is to rewrite the partition pruning code as an optimizer transformation which gets kicked
in after the predicate pushdown code. This refactor also uses the graph walker framework so
that the partition pruning code gets consolidated well with the frameworks and does not work
on the query block but rather works on the operator tree.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message