hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gautam <gautamkows...@gmail.com>
Subject Re: Push down hive filters to hbase store with composite rowkey ..
Date Tue, 08 Apr 2014 20:54:00 GMT
Any thoughts on this? Am I going down the right track? Is there an easier
workaround?

-Gautam.


On Mon, Apr 7, 2014 at 11:29 PM, Gautam <gautamkowshik@gmail.com> wrote:

> Hello All,
>               *Short version*: Looking to leverage hbase  fuzzy row scans
> on fields in a composite rowkey ( on hbase  0.94.6 ).
>
> *Long version*: Our hbase store design is heavily dependent on a bucketed
> composite rowkey that uses all the lovely features in hbase to spread the
> read/write workload across regions. We want to do  incremental processing
> using Hive queries (over hbase of course) on new data entering the system.
> It's of paramount importance to be able to reduce the scan size on each
> iteration of the HQL jobs.
>
>  We are running hbase 94.6 with Hive 0.10. Upgrading to a later version of
> hive is easy for us but upgrading Hbase will be much much harder. I have
> gone through the current docs under
> https://cwiki.apache.org/confluence/display/Hive/HBaseIntegration and
> related jiras viz. HIVE-6290, HIVE-6411 et al . Both those are patch
> available but possibly going into hive 0.13. Hive 0.13 is compatible with
> hbase 96 only I believe. My current plan is to embark on a patching
> expedition to try and retrofit these patches onto earlier versions of hive
> that play nice with hbase 94.6 But i reckon: "here be dragons".
>
> Ergo I am relying on the mighty hive user group to provide guidance in
> this hour of need. Please lead me and I shall follow.
>
>
> Cheers,
> -Gautam.
>
> ====
>
> *Footnote*: Our rowkey is a fixed length key(no delimiter) that looks
> like:  [bucket][App][MsgType][TimeSlice] ....
>
>
>


-- 
"If you really want something in this life, you have to work for it. Now,
quiet! They're about to announce the lottery numbers..."

Mime
View raw message