Mailing-List: contact dev-help@hive.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hive.apache.org
Date: Thu, 16 Jun 2011 18:05:50 +0000 (UTC)
From: "jiraposter@reviews.apache.org (JIRA)" <jira@apache.org>
To: hive-dev@hadoop.apache.org
Message-ID: 
 <1749916489.11860.1308247550688.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <1733304418.7230.1299653280020.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Commented] (HIVE-2036) Update bitmap indexes for automatic
 usage
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HIVE-2036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13050610#comment-13050610 ] 

jiraposter@reviews.apache.org commented on HIVE-2036:
-----------------------------------------------------


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/857/#review856
-----------------------------------------------------------


ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/index/IndexWhereTaskDispatcher.java
<https://reviews.apache.org/r/857/#comment1865>

    Need to update this comment now, explaining why we don't even look for the filter operator any more.


- John


On 2011-06-15 23:46:24, Syed Albiz wrote:
bq.  
bq.  -----------------------------------------------------------
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/857/
bq.  -----------------------------------------------------------
bq.  
bq.  (Updated 2011-06-15 23:46:24)
bq.  
bq.  
bq.  Review request for hive and John Sichi.
bq.  
bq.  
bq.  Summary
bq.  -------
bq.  
bq.  Add support for generating index queries to support automatic usage of bitmap indexes. This required changing the interface to the IndexHandlers to support accepting queries on multiple indexes. The compact indexes were modified to use this new interface as well, although no functional changes were made to how they work. Only supports AND predicates right now, but it should be possibly to extend the BitmapQuery interface defined in this patch to easily support OR predicates as well. Currently benchmarking these changes on a test cluster.
bq.  
bq.  
bq.  This addresses bug HIVE-2036.
bq.      https://issues.apache.org/jira/browse/HIVE-2036
bq.  
bq.  
bq.  Diffs
bq.  -----
bq.  
bq.    ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java 4fba845 
bq.    ql/src/java/org/apache/hadoop/hive/ql/index/HiveIndexHandler.java e5ee183 
bq.    ql/src/java/org/apache/hadoop/hive/ql/index/IndexPredicateAnalyzer.java 3caa4cc 
bq.    ql/src/java/org/apache/hadoop/hive/ql/index/bitmap/BitmapIndexHandler.java af9d7b1 
bq.    ql/src/java/org/apache/hadoop/hive/ql/index/bitmap/BitmapInnerQuery.java PRE-CREATION 
bq.    ql/src/java/org/apache/hadoop/hive/ql/index/bitmap/BitmapOuterQuery.java PRE-CREATION 
bq.    ql/src/java/org/apache/hadoop/hive/ql/index/bitmap/BitmapQuery.java PRE-CREATION 
bq.    ql/src/java/org/apache/hadoop/hive/ql/index/compact/CompactIndexHandler.java 56e7609 
bq.    ql/src/java/org/apache/hadoop/hive/ql/io/HiveFileFormatUtils.java d64e88b 
bq.    ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/index/IndexWhereProcessor.java 268560d 
bq.    ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/index/IndexWhereTaskDispatcher.java 0873e1a 
bq.    ql/src/java/org/apache/hadoop/hive/ql/ppd/ExprWalkerProcFactory.java 95fef73 
bq.    ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java d22654b 
bq.    ql/src/test/queries/clientpositive/index_auto_partitioned.q 5f92f04 
bq.    ql/src/test/queries/clientpositive/index_bitmap_auto.q PRE-CREATION 
bq.    ql/src/test/queries/clientpositive/index_bitmap_auto_partitioned.q PRE-CREATION 
bq.    ql/src/test/results/clientpositive/index_auto.q.out 713bb40 
bq.    ql/src/test/results/clientpositive/index_auto_file_format.q.out 894a556 
bq.    ql/src/test/results/clientpositive/index_auto_multiple.q.out 27092dc 
bq.    ql/src/test/results/clientpositive/index_auto_partitioned.q.out 05cc84a 
bq.    ql/src/test/results/clientpositive/index_auto_unused.q.out 8a1eda5 
bq.    ql/src/test/results/clientpositive/index_bitmap3.q.out dadfa77 
bq.    ql/src/test/results/clientpositive/index_bitmap_auto.q.out PRE-CREATION 
bq.    ql/src/test/results/clientpositive/index_bitmap_auto_partitioned.q.out PRE-CREATION 
bq.  
bq.  Diff: https://reviews.apache.org/r/857/diff
bq.  
bq.  
bq.  Testing
bq.  -------
bq.  
bq.  Passes unit tests, additional testcase to test automatic bitmap indexing index_bitmap_auto.q was also added to the TestCliDriver suite. Currently benchmarking changes on a test cluster.
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Syed
bq.  
bq.


> Update bitmap indexes for automatic usage
> -----------------------------------------
>
>                 Key: HIVE-2036
>                 URL: https://issues.apache.org/jira/browse/HIVE-2036
>             Project: Hive
>          Issue Type: Improvement
>          Components: Indexing
>    Affects Versions: 0.8.0
>            Reporter: Russell Melick
>            Assignee: Syed S. Albiz
>         Attachments: HIVE-2036.1.patch, HIVE-2036.3.patch
>
>
> HIVE-1644 will provide automatic usage of indexes, and HIVE-1803 adds bitmap index support.  The bitmap code will need to be extended after it is committed to enable automatic use of indexing.  Most work will be focused in the BitmapIndexHandler, which needs to generate the re-entrant QL index query.  There may also be significant work in the IndexPredicateAnalyzer to support predicates with OR's, instead of just AND's as it is currently.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira