hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "He Yongqiang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-1644) use filter pushdown for automatically accessing indexes
Date Mon, 21 Mar 2011 18:25:05 GMT

    [ https://issues.apache.org/jira/browse/HIVE-1644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13009261#comment-13009261
] 

He Yongqiang commented on HIVE-1644:
------------------------------------

a few comments:
rename work.getInputFormatFile to work.getIndexInputFile() or IndexIntermediateFile. and remove
LOG from IndexWhereResolver

IndexWhereTaskDispatcher:
findTableScanOps in IndexWhereTaskDispatcher is empty.
indexesOnTable in IndexWhereTaskDispatcher should be mapper<table, list<index>>
because there could be more than one table scanned in one task.
In getIndexes, use -1 instead of 1024

The reason of duplicate plan is because today's hive apply filter twice, you can verify that
by a simple "explain select key from src where key=86;". This is to be fixed in https://issues.apache.org/jira/browse/HIVE-1538.
So i guess what you can process the task only one time by remembering it in the IndexWhereProcCtx.

And i noticed that the patch added all new tasks as root tasks, but keep the child task (the
old root task) remain in root task. That may cause problem. So i guess the old task can just
be removed from root task once a new parent task is added to root task.

> use filter pushdown for automatically accessing indexes
> -------------------------------------------------------
>
>                 Key: HIVE-1644
>                 URL: https://issues.apache.org/jira/browse/HIVE-1644
>             Project: Hive
>          Issue Type: Improvement
>          Components: Indexing
>    Affects Versions: 0.7.0
>            Reporter: John Sichi
>            Assignee: Russell Melick
>         Attachments: HIVE-1644.1.patch, HIVE-1644.2.patch, HIVE-1644.3.patch, HIVE-1644.4.patch,
HIVE-1644.5.patch, HIVE-1644.6.patch, HIVE-1644.7.patch, HIVE-1644.8.patch, HIVE-1644.9.patch
>
>
> HIVE-1226 provides utilities for analyzing filters which have been pushed down to a table
scan.  The next step is to use these for selecting available indexes and generating access
plans for those indexes.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message