hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From " (JIRA)" <>
Subject [jira] [Commented] (HIVE-2128) Automatic Indexing with multiple tables
Date Wed, 13 Jul 2011 00:31:02 GMT

] commented on HIVE-2128:

This is an automatically generated e-mail. To reply, visit:

(Updated 2011-07-13 00:29:56.738368)

Review request for hive and John Sichi.


Revamped approach. We already uniquely assign filenames to each index query result, so instead
of throwing those away, keep them in the indexIntermediateFile variable, and take the union
of those input paths to generate the next set of input splits.


Grab the indexed tables during optimized query generation, grab the associated path URIs,
and keep those around in the Configuration object. When the job is passed to ExecDriver, this
data is extracted and used in HiveIndexedInputFormat to decide whether to use the index file
or delegate to the parent (HiveInputFormat) class. Not sure if this is robust. 

This addresses bug HIVE-2128.

Diffs (updated)

  ql/src/test/results/clientpositive/index_auto_self_join.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/index_auto_mult_tables_compact.q.out PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/ b278ffe 
  ql/src/java/org/apache/hadoop/hive/ql/index/ 617723e 
  ql/src/java/org/apache/hadoop/hive/ql/index/ b9b586e 
  ql/src/java/org/apache/hadoop/hive/ql/index/ f1ee95d 
  ql/src/java/org/apache/hadoop/hive/ql/index/ 02ab78c 
  ql/src/java/org/apache/hadoop/hive/ql/index/bitmap/ 61bbbf5 
  ql/src/java/org/apache/hadoop/hive/ql/index/compact/ 7c91946 
  ql/src/java/org/apache/hadoop/hive/ql/plan/ a03a9a6 
  ql/src/test/queries/clientpositive/index_auto_mult_tables.q PRE-CREATION 
  ql/src/test/queries/clientpositive/index_auto_mult_tables_compact.q PRE-CREATION 
  ql/src/test/queries/clientpositive/index_auto_self_join.q PRE-CREATION 
  ql/src/test/results/clientpositive/index_auto_mult_tables.q.out PRE-CREATION 



added new testcase index_auto_mult_tables.q



> Automatic Indexing with multiple tables
> ---------------------------------------
>                 Key: HIVE-2128
>                 URL:
>             Project: Hive
>          Issue Type: Improvement
>          Components: Indexing
>    Affects Versions: 0.8.0
>            Reporter: Russell Melick
>            Assignee: Syed S. Albiz
>         Attachments: HIVE-2128.1.patch, HIVE-2128.1.patch, HIVE-2128.2.patch
> Make automatic indexing work with jobs which access multiple tables.  We'll probably
need to modify the way that the index input format works in order to associate index formats/files
with specific tables.

This message is automatically generated by JIRA.
For more information on JIRA, see:


View raw message