hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Mains <andrew.ma...@kontagent.com>
Subject Re: Predicate pushdown on HBase snapshots
Date Thu, 02 Apr 2015 23:48:20 GMT

> Are you suggesting taking advantage of the sorted order to seek to the key
mentioned in a SARG

Pretty much, yes. It's essentially the same use case as predicate 
pushdown for the live table case (already implemented), which converts 
predicates into a scan, and we should be able to reuse a significant 
amount of that code. It is perhaps a somewhat limited use case, but I'd 
argue that it's a reasonably significant one for hive on HBase--if 
you've designed your HBase row key based on your query patterns, it's 
reasonable to expect that most queries over snapshots will be SARGable 
(that's certainly true for our use case, though I can't speak so much to 
others).

Given that, does it seem worthwhile enough to file a ticket? We may 
implement it either way (depending on how our preliminary performance 
testing of queries over snapshots goes).

Thanks!

Andrew

On 3/30/15 8:03 PM, Gopal Vijayaraghavan wrote:
>> Looking at the current implementation on trunk, hive's hbase integration
>> doesn't currently seem to support predicate pushdown for queries over
>> HBase snapshots. Does this seem like a reasonable feature to add?
>> It would be nice to have relative feature parity between queries running
>> over snapshots and queries running over live tables.
> Are you suggesting taking advantage of the sorted order to seek to the key
> mentioned in a SARG?
>
> That particular method will be limited to simple filters on exactly one
> key or perhaps with a few seeks, the more generic IN/BETWEEN SARGs.
>
> But for that case, it will provided a significant boost.
>
> Cheers,
> Gopal
>
>


Mime
View raw message