hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "John Sichi" <jsi...@fb.com>
Subject Re: Review Request: HIVE-2036: Update bitmap indexes for automatic usage
Date Wed, 08 Jun 2011 23:32:57 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/857/#review785
-----------------------------------------------------------



ql/src/java/org/apache/hadoop/hive/ql/index/HiveIndexHandler.java
<https://reviews.apache.org/r/857/#comment1681>

    I think that should be a period instead of a comma in "indexes, if"



ql/src/java/org/apache/hadoop/hive/ql/index/HiveIndexHandler.java
<https://reviews.apache.org/r/857/#comment1682>

    How exactly are they combined?  This Javadoc should be written as a contract between the
optimizer and the index plugin author, so that the author knows exactly how to interpret the
inputs and also what is going to be done with the output.
    



ql/src/java/org/apache/hadoop/hive/ql/index/bitmap/BitmapIndexHandler.java
<https://reviews.apache.org/r/857/#comment1683>

    Why do you need to use toArray here?  indexCols.keySet is already a collection.



ql/src/java/org/apache/hadoop/hive/ql/index/bitmap/BitmapIndexHandler.java
<https://reviews.apache.org/r/857/#comment1684>

    Why are you converting the search conditions back into predicate form here?  Wouldn't
it be easier to analyze them as search conditions?


- John


On 2011-06-08 00:22:37, Syed Albiz wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/857/
> -----------------------------------------------------------
> 
> (Updated 2011-06-08 00:22:37)
> 
> 
> Review request for hive and John Sichi.
> 
> 
> Summary
> -------
> 
> Add support for generating index queries to support automatic usage of bitmap indexes.
This required changing the interface to the IndexHandlers to support accepting queries on
multiple indexes. The compact indexes were modified to use this new interface as well, although
no functional changes were made to how they work. Only supports AND predicates right now,
but it should be possibly to extend the BitmapQuery interface defined in this patch to easily
support OR predicates as well. Currently benchmarking these changes on a test cluster.
> 
> 
> This addresses bug HIVE-2036.
>     https://issues.apache.org/jira/browse/HIVE-2036
> 
> 
> Diffs
> -----
> 
>   ql/src/test/queries/clientpositive/index_bitmap_auto_partitioned.q PRE-CREATION 
>   ql/src/test/results/clientpositive/index_bitmap_auto.q.out PRE-CREATION 
>   ql/src/test/queries/clientpositive/index_bitmap_auto.q PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/index/IndexWhereTaskDispatcher.java
0873e1a 
>   ql/src/java/org/apache/hadoop/hive/ql/index/compact/CompactIndexHandler.java 56e7609

>   ql/src/java/org/apache/hadoop/hive/ql/io/HiveFileFormatUtils.java d64e88b 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/index/IndexWhereProcessor.java
268560d 
>   ql/src/java/org/apache/hadoop/hive/ql/index/bitmap/BitmapQuery.java PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/index/bitmap/BitmapInnerQuery.java PRE-CREATION

>   ql/src/java/org/apache/hadoop/hive/ql/index/bitmap/BitmapOuterQuery.java PRE-CREATION

>   ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java 4fba845 
>   ql/src/java/org/apache/hadoop/hive/ql/index/HiveIndexHandler.java e5ee183 
>   ql/src/java/org/apache/hadoop/hive/ql/index/bitmap/BitmapIndexHandler.java af9d7b1

>   ql/src/test/results/clientpositive/index_bitmap_auto_partitioned.q.out PRE-CREATION

> 
> Diff: https://reviews.apache.org/r/857/diff
> 
> 
> Testing
> -------
> 
> Passes unit tests, additional testcase to test automatic bitmap indexing index_bitmap_auto.q
was also added to the TestCliDriver suite. Currently benchmarking changes on a test cluster.
> 
> 
> Thanks,
> 
> Syed
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message