hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Namit Jain (JIRA)" <>
Subject [jira] Commented: (HIVE-1694) Accelerate query execution using indexes
Date Mon, 01 Nov 2010 18:20:26 GMT


Namit Jain commented on HIVE-1694:

Had an offline discussion with John just now - I think he is also giving similar comments,
so I will keep it very brief.

One concern is that all the optimizations should be at the operator level, I mean - this should
be treated as just another optimization.
As you mentioned in your presentation correctly, Hive does not support cost-based optimizer
currently, and it will 
require all the optimizations to be consolidated in one place to help move to that model.

We are thinking about moving the group by skews also in the optimizer (instead of the current
approach in
SemanticAnalyzer). Once all the optimizations are in a central place, it will be much easier
to move to costing.

The Harvey Mudd folks currently are not looking at Group By optimizations for indexing, so,
this will be extremely
useful for the whole community.

> Accelerate query execution using indexes
> ----------------------------------------
>                 Key: HIVE-1694
>                 URL:
>             Project: Hive
>          Issue Type: New Feature
>          Components: Indexing, Query Processor
>    Affects Versions: 0.7.0
>            Reporter: Nikhil Deshpande
>         Attachments: demo_q1.hql, demo_q2.hql, HIVE-1694_2010-10-28.diff
> The index building patch (Hive-417) is checked into trunk, this JIRA issue tracks supporting
indexes in Hive compiler & execution engine for SELECT queries.
> This is in ref. to John's comment at
> on creating separate JIRA issue for tracking index usage in optimizer & query execution.
> The aim of this effort is to use indexes to accelerate query execution (for certain class
of queries). E.g.
> - Filters and range scans (already being worked on by He Yongqiang as part of HIVE-417?)
> - Joins (index based joins)
> - Group By, Order By and other misc cases
> The proposal is multi-step:
> 1. Building index based operators, compiler and execution engine changes
> 2. Optimizer enhancements (e.g. cost-based optimizer to compare and choose between index
scans, full table scans etc.)
> This JIRA initially focuses on the first step. This JIRA is expected to hold the information
about index based plans & operator implementations for above mentioned cases. 

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message