Return-Path: X-Original-To: apmail-hive-dev-archive@www.apache.org Delivered-To: apmail-hive-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id D807E6C00 for ; Mon, 13 Jun 2011 22:57:44 +0000 (UTC) Received: (qmail 58908 invoked by uid 500); 13 Jun 2011 22:57:44 -0000 Delivered-To: apmail-hive-dev-archive@hive.apache.org Received: (qmail 58839 invoked by uid 500); 13 Jun 2011 22:57:44 -0000 Mailing-List: contact dev-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hive.apache.org Delivered-To: mailing list dev@hive.apache.org Received: (qmail 58829 invoked by uid 99); 13 Jun 2011 22:57:44 -0000 Received: from reviews.apache.org (HELO reviews.apache.org) (140.211.11.40) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 13 Jun 2011 22:57:44 +0000 Received: from reviews.apache.org (localhost [127.0.0.1]) by reviews.apache.org (Postfix) with ESMTP id 91F9F1C002A; Mon, 13 Jun 2011 22:57:46 +0000 (UTC) Content-Type: multipart/alternative; boundary="===============0361824823676758044==" MIME-Version: 1.0 Subject: Re: Review Request: HIVE-2036: Update bitmap indexes for automatic usage From: "John Sichi" To: "John Sichi" Date: Mon, 13 Jun 2011 22:57:46 -0000 Message-ID: <20110613225746.13642.4733@reviews.apache.org> X-ReviewBoard-URL: https://reviews.apache.org X-ReviewRequest-URL: https://reviews.apache.org/r/857/ Cc: "Syed Albiz" ,"hive" In-Reply-To: <20110611190542.30868.88881@reviews.apache.org> References: <20110611190542.30868.88881@reviews.apache.org> --===============0361824823676758044== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/857/#review825 ----------------------------------------------------------- ql/src/java/org/apache/hadoop/hive/ql/index/IndexPredicateAnalyzer.java I don't think this should be necessary. We just want to propagate the = partition column predicate (whatever it is) from the base table query to th= e index table query; partition pruning on the index table query will do the= rest of the work. = In other words, if the original query had = part_key=3D = we want to preserve that on the index table query. That's what the cod= e is already supposed to be doing before your change; was it not working? = - John On 2011-06-11 19:05:42, Syed Albiz wrote: > = > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/857/ > ----------------------------------------------------------- > = > (Updated 2011-06-11 19:05:42) > = > = > Review request for hive and John Sichi. > = > = > Summary > ------- > = > Add support for generating index queries to support automatic usage of bi= tmap indexes. This required changing the interface to the IndexHandlers to = support accepting queries on multiple indexes. The compact indexes were mod= ified to use this new interface as well, although no functional changes wer= e made to how they work. Only supports AND predicates right now, but it sho= uld be possibly to extend the BitmapQuery interface defined in this patch t= o easily support OR predicates as well. Currently benchmarking these change= s on a test cluster. > = > = > This addresses bug HIVE-2036. > https://issues.apache.org/jira/browse/HIVE-2036 > = > = > Diffs > ----- > = > ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java 4fba845 = > ql/src/java/org/apache/hadoop/hive/ql/index/HiveIndexHandler.java e5ee1= 83 = > ql/src/java/org/apache/hadoop/hive/ql/index/IndexPredicateAnalyzer.java= 3caa4cc = > ql/src/java/org/apache/hadoop/hive/ql/index/bitmap/BitmapIndexHandler.j= ava af9d7b1 = > ql/src/java/org/apache/hadoop/hive/ql/index/bitmap/BitmapInnerQuery.jav= a PRE-CREATION = > ql/src/java/org/apache/hadoop/hive/ql/index/bitmap/BitmapOuterQuery.jav= a PRE-CREATION = > ql/src/java/org/apache/hadoop/hive/ql/index/bitmap/BitmapQuery.java PRE= -CREATION = > ql/src/java/org/apache/hadoop/hive/ql/index/compact/CompactIndexHandler= .java 56e7609 = > ql/src/java/org/apache/hadoop/hive/ql/io/HiveFileFormatUtils.java d64e8= 8b = > ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/index/IndexWhe= reProcessor.java 268560d = > ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/index/IndexWhe= reTaskDispatcher.java 0873e1a = > ql/src/test/queries/clientpositive/index_auto_partitioned.q 5f92f04 = > ql/src/test/queries/clientpositive/index_bitmap_auto.q PRE-CREATION = > ql/src/test/queries/clientpositive/index_bitmap_auto_partitioned.q PRE-= CREATION = > ql/src/test/results/clientpositive/index_auto_partitioned.q.out 05cc84a = > ql/src/test/results/clientpositive/index_bitmap_auto.q.out PRE-CREATION = > ql/src/test/results/clientpositive/index_bitmap_auto_partitioned.q.out = PRE-CREATION = > = > Diff: https://reviews.apache.org/r/857/diff > = > = > Testing > ------- > = > Passes unit tests, additional testcase to test automatic bitmap indexing = index_bitmap_auto.q was also added to the TestCliDriver suite. Currently be= nchmarking changes on a test cluster. > = > = > Thanks, > = > Syed > = > --===============0361824823676758044==--