hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "jiraposter@reviews.apache.org (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-2036) Update bitmap indexes for automatic usage
Date Wed, 15 Jun 2011 23:46:48 GMT

    [ https://issues.apache.org/jira/browse/HIVE-2036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13050150#comment-13050150
] 

jiraposter@reviews.apache.org commented on HIVE-2036:
-----------------------------------------------------


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/857/
-----------------------------------------------------------

(Updated 2011-06-15 23:46:24.176586)


Review request for hive and John Sichi.


Changes
-------

Used setFilterExpr on the TableScanDesc to propagate the complete original predicate as the
partition predicate was getting removed by the PartitionConditionRemover from the FilterOperator


Summary
-------

Add support for generating index queries to support automatic usage of bitmap indexes. This
required changing the interface to the IndexHandlers to support accepting queries on multiple
indexes. The compact indexes were modified to use this new interface as well, although no
functional changes were made to how they work. Only supports AND predicates right now, but
it should be possibly to extend the BitmapQuery interface defined in this patch to easily
support OR predicates as well. Currently benchmarking these changes on a test cluster.


This addresses bug HIVE-2036.
    https://issues.apache.org/jira/browse/HIVE-2036


Diffs (updated)
-----

  ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java 4fba845 
  ql/src/java/org/apache/hadoop/hive/ql/index/HiveIndexHandler.java e5ee183 
  ql/src/java/org/apache/hadoop/hive/ql/index/IndexPredicateAnalyzer.java 3caa4cc 
  ql/src/java/org/apache/hadoop/hive/ql/index/bitmap/BitmapIndexHandler.java af9d7b1 
  ql/src/java/org/apache/hadoop/hive/ql/index/bitmap/BitmapInnerQuery.java PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/index/bitmap/BitmapOuterQuery.java PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/index/bitmap/BitmapQuery.java PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/index/compact/CompactIndexHandler.java 56e7609 
  ql/src/java/org/apache/hadoop/hive/ql/io/HiveFileFormatUtils.java d64e88b 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/index/IndexWhereProcessor.java
268560d 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/index/IndexWhereTaskDispatcher.java
0873e1a 
  ql/src/java/org/apache/hadoop/hive/ql/ppd/ExprWalkerProcFactory.java 95fef73 
  ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java d22654b 
  ql/src/test/queries/clientpositive/index_auto_partitioned.q 5f92f04 
  ql/src/test/queries/clientpositive/index_bitmap_auto.q PRE-CREATION 
  ql/src/test/queries/clientpositive/index_bitmap_auto_partitioned.q PRE-CREATION 
  ql/src/test/results/clientpositive/index_auto.q.out 713bb40 
  ql/src/test/results/clientpositive/index_auto_file_format.q.out 894a556 
  ql/src/test/results/clientpositive/index_auto_multiple.q.out 27092dc 
  ql/src/test/results/clientpositive/index_auto_partitioned.q.out 05cc84a 
  ql/src/test/results/clientpositive/index_auto_unused.q.out 8a1eda5 
  ql/src/test/results/clientpositive/index_bitmap3.q.out dadfa77 
  ql/src/test/results/clientpositive/index_bitmap_auto.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/index_bitmap_auto_partitioned.q.out PRE-CREATION 

Diff: https://reviews.apache.org/r/857/diff


Testing
-------

Passes unit tests, additional testcase to test automatic bitmap indexing index_bitmap_auto.q
was also added to the TestCliDriver suite. Currently benchmarking changes on a test cluster.


Thanks,

Syed



> Update bitmap indexes for automatic usage
> -----------------------------------------
>
>                 Key: HIVE-2036
>                 URL: https://issues.apache.org/jira/browse/HIVE-2036
>             Project: Hive
>          Issue Type: Improvement
>          Components: Indexing
>    Affects Versions: 0.8.0
>            Reporter: Russell Melick
>            Assignee: Syed S. Albiz
>         Attachments: HIVE-2036.1.patch, HIVE-2036.3.patch
>
>
> HIVE-1644 will provide automatic usage of indexes, and HIVE-1803 adds bitmap index support.
 The bitmap code will need to be extended after it is committed to enable automatic use of
indexing.  Most work will be focused in the BitmapIndexHandler, which needs to generate the
re-entrant QL index query.  There may also be significant work in the IndexPredicateAnalyzer
to support predicates with OR's, instead of just AND's as it is currently.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message