hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ankit kinra <ankitki...@gmail.com>
Subject Re: Composite Key Handling in Hbase + Hive Integration
Date Wed, 25 Jul 2012 03:51:50 GMT
Hi  swarnim ,

Yes that workaround worked for me perfectly, but hive was having problems
reading the bytes of the first key (first part of key) which was a Long
stored as bytes using Bytes.toBytes(Long) of Hbase.
I tried using BIGINT for the part of the key but it did not work.
I guess I will search more on this, but the composite key solution worked
like a charm.
Thanks a lot.

Regards,
Ankit Kinra
Master of Computer Science, 2013
University of Minnesota, Twin Cities



On Tue, Jul 24, 2012 at 7:49 AM, kulkarni.swarnim@gmail.com <
kulkarni.swarnim@gmail.com> wrote:

> Try something like this:
>
> CREATE EXTERNAL TABLE hbase_table_1(key
> struct<a:string,b:string,c:string>, value string)
>
> ROW FORMAT DELIMITED
>
> COLLECTION ITEMS TERMINATED BY '~'
>
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
>
> WITH SERDEPROPERTIES ("hbase.columns.mapping" =
> ":key,test-family:test-qual")
>
> TBLPROPERTIES ("hbase.table.name" = "SIMPLE_TABLE");
>
> Basically what you are doing here is that you are visualizing the
> composite key as a struct and specifying that your keys in the composite
> key are separated by a "~". After doing this, to GROUP BY any key in your
> composite key, you simply run a query like:
>
> select * from hbase_table_2 GROUP BY key.a;
>
> This should give you your desired result.
>
> Let me know if this works for you. We can then add this as a workaround on
> that bug.
>
> On Tue, Jul 24, 2012 at 2:14 AM, ankit kinra <ankitkinra@gmail.com> wrote:
>
>> Hi,
>>
>> I have a use case in HBase + Hive Integration where HBase primary key is
>> a composite key and the keys is separated by us with a custom delimiter. So
>> basically it is Key = A~B~C.
>>  Now, I wanted to run a query on this HBase table using Hive and group by
>> "A" (and not the complete primary key). I went through the following
>> presentation :
>>
>> https://docs.google.com/viewer?a=v&q=cache:GHg9GMFOZVwJ:assets.en.oreilly.com/1/event/61/HBase%2520and%2520Hive%2520at%2520StumbleUpon%2520Presentation.ppt+hbase+composite+key+hive&hl=en&gl=us&pid=bl&srcid=ADGEEShTyoUXyvXptTu4pMjje_FkaN_j1OK9wG0lclWWsKNjGreLTkk3IDqT16xO8ClqIfzhM69aeU7Gph4kZPxTS-PXvLiWPSRvgS2WEjnvViPJhpM0ItsLaTWq1DRuUgOzKhjSzIlx&sig=AHIEtbT4scO3IdtvLYG3RtLoKN5gG1udPg
>>
>> It says that this was implemented at StumbleUpon, anybody having any idea
>> if that can be used by others.
>>
>> Also, there is this issue in JIRA :
>> https://issues.apache.org/jira/browse/HIVE-2599 which talks about
>> similar feature.
>>
>> So it would be very helpful if anyone can give me some idea regarding
>> this.
>>
>> Regards,
>> Ankit Kinra
>>
>>
>
>
> --
> Swarnim
>

Mime
View raw message