hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Syed Albiz" <>
Subject Re: Review Request: HIVE-2036: Update bitmap indexes for automatic usage
Date Tue, 14 Jun 2011 04:05:43 GMT

This is an automatically generated e-mail. To reply, visit:

(Updated 2011-06-14 04:05:43.158797)

Review request for hive and John Sichi.


Removed redundant check on partition predicate (which is done in IndexWhereProcessor). The
reason this was causing problems was that when the index was being built, the query generated
to build the index was run through the optimizer and at this stage the optimizer thought that
the index was already built and had the partition. A simpler solution is to just disable index
query optimization for building indexes. 


Add support for generating index queries to support automatic usage of bitmap indexes. This
required changing the interface to the IndexHandlers to support accepting queries on multiple
indexes. The compact indexes were modified to use this new interface as well, although no
functional changes were made to how they work. Only supports AND predicates right now, but
it should be possibly to extend the BitmapQuery interface defined in this patch to easily
support OR predicates as well. Currently benchmarking these changes on a test cluster.

This addresses bug HIVE-2036.

Diffs (updated)

  ql/src/java/org/apache/hadoop/hive/ql/exec/ 4fba845 
  ql/src/java/org/apache/hadoop/hive/ql/index/ e5ee183 
  ql/src/java/org/apache/hadoop/hive/ql/index/ 3caa4cc 
  ql/src/java/org/apache/hadoop/hive/ql/index/bitmap/ af9d7b1 
  ql/src/java/org/apache/hadoop/hive/ql/index/bitmap/ PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/index/bitmap/ PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/index/bitmap/ PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/index/compact/ 56e7609 
  ql/src/java/org/apache/hadoop/hive/ql/io/ d64e88b 
  ql/src/test/queries/clientpositive/index_auto_partitioned.q 5f92f04 
  ql/src/test/queries/clientpositive/index_bitmap_auto.q PRE-CREATION 
  ql/src/test/queries/clientpositive/index_bitmap_auto_partitioned.q PRE-CREATION 
  ql/src/test/results/clientpositive/index_auto_partitioned.q.out 05cc84a 
  ql/src/test/results/clientpositive/index_bitmap_auto.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/index_bitmap_auto_partitioned.q.out PRE-CREATION 



Passes unit tests, additional testcase to test automatic bitmap indexing index_bitmap_auto.q
was also added to the TestCliDriver suite. Currently benchmarking changes on a test cluster.



  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message