hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "He Yongqiang (JIRA)" <>
Subject [jira] [Commented] (HIVE-1644) use filter pushdown for automatically accessing indexes
Date Mon, 21 Mar 2011 18:25:05 GMT


He Yongqiang commented on HIVE-1644:

a few comments:
rename work.getInputFormatFile to work.getIndexInputFile() or IndexIntermediateFile. and remove
LOG from IndexWhereResolver

findTableScanOps in IndexWhereTaskDispatcher is empty.
indexesOnTable in IndexWhereTaskDispatcher should be mapper<table, list<index>>
because there could be more than one table scanned in one task.
In getIndexes, use -1 instead of 1024

The reason of duplicate plan is because today's hive apply filter twice, you can verify that
by a simple "explain select key from src where key=86;". This is to be fixed in
So i guess what you can process the task only one time by remembering it in the IndexWhereProcCtx.

And i noticed that the patch added all new tasks as root tasks, but keep the child task (the
old root task) remain in root task. That may cause problem. So i guess the old task can just
be removed from root task once a new parent task is added to root task.

> use filter pushdown for automatically accessing indexes
> -------------------------------------------------------
>                 Key: HIVE-1644
>                 URL:
>             Project: Hive
>          Issue Type: Improvement
>          Components: Indexing
>    Affects Versions: 0.7.0
>            Reporter: John Sichi
>            Assignee: Russell Melick
>         Attachments: HIVE-1644.1.patch, HIVE-1644.2.patch, HIVE-1644.3.patch, HIVE-1644.4.patch,
HIVE-1644.5.patch, HIVE-1644.6.patch, HIVE-1644.7.patch, HIVE-1644.8.patch, HIVE-1644.9.patch
> HIVE-1226 provides utilities for analyzing filters which have been pushed down to a table
scan.  The next step is to use these for selecting available indexes and generating access
plans for those indexes.

This message is automatically generated by JIRA.
For more information on JIRA, see:

View raw message