hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Russell Melick" <rmel...@cs.hmc.edu>
Subject Re: Review Request: HIVE-1644 Use filter pushdown for automatically accessing indexes
Date Fri, 15 Apr 2011 08:19:15 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/558/#review472
-----------------------------------------------------------



ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/index/IndexWhereProcessor.java
<https://reviews.apache.org/r/558/#comment953>

    I would have liked to just make a copy of pctx before I called rewriteForIndex(...) for
every index, and then just use whichever of those corresponded to the index rewrite we chose.
 However, the pctx did not seem to have an easy way to copy it.



ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/index/IndexWhereProcessor.java
<https://reviews.apache.org/r/558/#comment957>

    Do we need to propagate the residual predicate any further?



ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/index/IndexWhereProcessor.java
<https://reviews.apache.org/r/558/#comment955>

    I'm kind of confused about how to check the actual table and not the metadata.  When we
call indexTable.getPartitionKeys() and part.getTable.getPartitionKeys(), that method calls
getPartitionKeys() on the underlying Thrift Tables.  Is there a way besides getPartitionKeys()
that we should be using?



ql/src/test/queries/clientpositive/index_opt_where.q
<https://reviews.apache.org/r/558/#comment956>

    I have not yet added the additional unit tests



ql/src/test/results/clientpositive/index_opt_where_partitioned.q.out
<https://reviews.apache.org/r/558/#comment954>

    I fixed the labeling for this case, but would it make sense to label our stages differently
for indexing?  We only relabel correctly as long as we're overwriting the highest numbered
stage, since we only relabel a single task.  Or, should it relabel all tasks in the whole
plan?  We only have easy access to the context.currentTask when we iterate through in IndexWhereProcessor
(line 153)


- Russell


On 2011-04-15 08:08:14, Russell Melick wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/558/
> -----------------------------------------------------------
> 
> (Updated 2011-04-15 08:08:14)
> 
> 
> Review request for hive.
> 
> 
> Summary
> -------
> 
> Review request for HIVE-1644.12.patch
> 
> 
> This addresses bug HIVE-1644.
>     https://issues.apache.org/jira/browse/HIVE-1644
> 
> 
> Diffs
> -----
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java a21f589 
>   conf/hive-default.xml c42197f 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java 6437385 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/TableScanOperator.java c02d90b 
>   ql/src/java/org/apache/hadoop/hive/ql/index/AbstractIndexHandler.java dd0186d 
>   ql/src/java/org/apache/hadoop/hive/ql/index/HiveIndexHandler.java 411b78f 
>   ql/src/java/org/apache/hadoop/hive/ql/index/compact/CompactIndexHandler.java 1f01446

>   ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java 50db44c 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMRTableScan1.java 6162676 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/IndexWhereResolver.java PRE-CREATION

>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/PhysicalOptimizer.java 0ae9fa2

>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/index/IndexWhereProcCtx.java
PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/index/IndexWhereProcessor.java
PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/index/IndexWhereTaskDispatcher.java
PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/ParseContext.java 937a7b3 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java f0aca84 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/MapredWork.java 73391e9 
>   ql/src/test/queries/clientpositive/index_opt_where.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/index_opt_where_partitioned.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/index_opt_where_simple.q PRE-CREATION 
>   ql/src/test/results/clientpositive/index_opt_where.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/index_opt_where_partitioned.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/index_opt_where_simple.q.out PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/558/diff
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Russell
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message