hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "John Sichi (Commented) (JIRA)" <>
Subject [jira] [Commented] (HIVE-1643) support range scans and non-key columns in HBase filter pushdown
Date Tue, 27 Sep 2011 23:06:45 GMT


John Sichi commented on HIVE-1643:

Sandy, it would be great if you and Vaibhav can push this one through together.

I'm not sure what you mean by "striking" HiveStoragePredicateHandler. Its purpose is to allow
handlers to tell Hive which portion of the predicate got pushed down, and which part needs
to be handled by Hive during per-row evaluation.  This helps reduce the mapper work, so we
do not want to lose that.

IndexPredicateAnalyzer currently only handles conjunction (AND), so for your example, it would
only extract the condition for "key >= b" (and leave the right-hand side of the AND to
be evaluated by Hive).  Vacuous cases would be better handled generically inside of Hive's
optimizer (as part of constant folding), rather than specifically in index predicate analysis.

Note that for non-key columns, youll need to change newIndexPredicateAnalyzer to not call

Also watch out for the comment "We'll need to handle this better later when we support more
interesting predicates" in HBaseStorageHandler.decomposePredicate.  :)

> support range scans and non-key columns in HBase filter pushdown
> ----------------------------------------------------------------
>                 Key: HIVE-1643
>                 URL:
>             Project: Hive
>          Issue Type: Improvement
>          Components: HBase Handler
>    Affects Versions: 0.7.0
>            Reporter: John Sichi
>            Assignee: Vaibhav Aggarwal
> HIVE-1226 added support for WHERE rowkey=3.  We would like to support WHERE rowkey BETWEEN
10 and 20, as well as predicates on non-rowkeys (plus conjunctions etc).  Non-rowkey conditions
can't be used to filter out entire ranges, but they can be used to push the per-row filter
processing as far down as possible.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


View raw message