hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Swarnim Kulkarni (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-2599) Support Composit/Compound Keys with HBaseStorageHandler
Date Sat, 04 Jan 2014 07:12:51 GMT

    [ https://issues.apache.org/jira/browse/HIVE-2599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13862231#comment-13862231
] 

Swarnim Kulkarni commented on HIVE-2599:
----------------------------------------

Attached is the latest patch rebased with the master. The patch should apply cleanly now.

[~brocknoland] On your question about inserts, I think I might have misunderstood you a little
bit. I ran the following queries to test inserts on composite keys and was able to do it successfully.

{noformat}
CREATE EXTERNAL TABLE test_table_1(key struct<personId:string,value:string>, data string)

ROW FORMAT SERDE 'org.apache.hadoop.hive.hbase.HBaseSerDe' 
STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' 
WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf:value) 
TBLPROPERTIES ("hbase.table.name" = "hbase_test_table_1","hbase.composite.key.class"="com.test.hive.TestHBaseCompositeKey")

select * from test_table_1;

{"personid":"person1","value":"value1"}	1385435417948
{"personid":"person2","value":"value2"}	1386691798261
{"personid":"person3","value":"value3"}	1387481795304
{"personid":"person4","value":"value4"}	1386705359123
{"personid":"person5","value":"value5"}	1386972894836
......

CREATE TABLE test_table_2(key struct<personId:string,value:string>, value string) 
ROW FORMAT SERDE 'org.apache.hadoop.hive.hbase.HBaseSerDe'
STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val")
TBLPROPERTIES ("hbase.table.name" = "hbase_test_table_2");

INSERT OVERWRITE TABLE test_table_2 select key,data from test_table_1;
14/01/04 00:32:33 INFO ql.Driver: Launching Job 1 out of 1
14/01/04 00:32:58 INFO exec.Task: 2014-01-04 00:32:58,720 Stage-0 map = 0%,  reduce = 0%
....
2014-01-04 00:33:29,930 Stage-0 map = 100%,  reduce = 100%, Cumulative CPU 5.48 sec

select * from test_table_2;

{"personid":"person1","value":"value1"}	1385435417948
{"personid":"person2","value":"value2"}	1386691798261
{"personid":"person3","value":"value3"}	1387481795304
{"personid":"person4","value":"value4"}	1386705359123
{"personid":"person5","value":"value5"}	1386972894836
......
{noformat}

If this is what you meant, then yes the patch will handle both select and inserts. If not,
then please let me know so that I will log a new bug and tackle it accordingly.

> Support Composit/Compound Keys with HBaseStorageHandler
> -------------------------------------------------------
>
>                 Key: HIVE-2599
>                 URL: https://issues.apache.org/jira/browse/HIVE-2599
>             Project: Hive
>          Issue Type: Improvement
>          Components: HBase Handler
>    Affects Versions: 0.8.0
>            Reporter: Hans Uhlig
>            Assignee: Swarnim Kulkarni
>         Attachments: HIVE-2599.1.patch.txt, HIVE-2599.2.patch.txt, HIVE-2599.2.patch.txt
>
>
> It would be really nice for hive to be able to understand composite keys from an underlying
HBase schema. Currently we have to store key fields twice to be able to both key and make
data available. I noticed John Sichi mentioned in HIVE-1228 that this would be a separate
issue but I cant find any follow up. How feasible is this in the HBaseStorageHandler?



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message