hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kula Liao (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HIVE-705) Let Hive can analyse hbase's tables
Date Thu, 13 Aug 2009 03:50:14 GMT

    [ https://issues.apache.org/jira/browse/HIVE-705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12742697#action_12742697
] 

Kula Liao commented on HIVE-705:
--------------------------------

Hi Samuel,

Thanks for your great job.
I found some error when testing your patch.

The sql statements are from the file : "ql/src/test/queries/clienthbase/hbase_queries.q".
I created a table named "hbase_table_1" using the following statement:

CREATE TABLE hbase_table_1(key int, value string) 
ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.hbase.HBaseSerDe' 
WITH SERDEPROPERTIES (
"hbase.columns.mapping" = "cf:string"
) STORED AS HBASETABLE;

OK. Then I inserted data into "hbase_table_1". 

hive> FROM src INSERT OVERWRITE TABLE hbase_table_1 SELECT *;
Total MapReduce jobs = 1
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_200908131113_0002, Tracking URL = http://localhost:50030/jobdetails.jsp?jobid=job_200908131113_0002
Kill Command = /home/stephen/hadoop-0.19.2/bin/../bin/hadoop job  -Dmapred.job.tracker=localhost:9001
-kill job_200908131113_0002
2009-08-13 11:17:07,162 map = 0%,  reduce =0%
2009-08-13 11:17:14,200 map = 50%,  reduce =0%
2009-08-13 11:17:15,215 map = 100%,  reduce =0%
Ended Job = job_200908131113_0002
500 Rows loaded to hbase_table_1
OK

When I tried to do some queries. I found the following error message:

hive> select * from hbase_table_1 where value > '0';         
Total MapReduce jobs = 1
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_200908131113_0003, Tracking URL = http://localhost:50030/jobdetails.jsp?jobid=job_200908131113_0003
Kill Command = /home/stephen/hadoop-0.19.2/bin/../bin/hadoop job  -Dmapred.job.tracker=localhost:9001
-kill job_200908131113_0003
2009-08-13 11:18:24,019 map = 0%,  reduce =0%
2009-08-13 11:18:42,146 map = 100%,  reduce =100%
Ended Job = job_200908131113_0003 with errors
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.ExecDriver

The following message is found in the mapreduce log:

java.lang.NullPointerException
	at org.apache.hadoop.hbase.mapred.TableInputFormat.configure(TableInputFormat.java:52)
	at org.apache.hadoop.hive.ql.io.HiveHBaseTableInputFormat.configure(HiveHBaseTableInputFormat.java:36)
	at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
	at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:83)
	at org.apache.hadoop.hive.ql.io.HiveInputFormat.getInputFormatFromCache(HiveInputFormat.java:184)
	at org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:211)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:331)
	at org.apache.hadoop.mapred.Child.main(Child.java:158)

There is another query, nothing returned.
hive> select * from hbase_table_1;
OK
Time taken: 2.952 seconds

> Let Hive can analyse hbase's tables
> -----------------------------------
>
>                 Key: HIVE-705
>                 URL: https://issues.apache.org/jira/browse/HIVE-705
>             Project: Hadoop Hive
>          Issue Type: New Feature
>            Reporter: Samuel Guo
>         Attachments: hbase-0.19.3-test.jar, hbase-0.19.3.jar, HIVE-705_draft.patch
>
>
> Add a serde over the hbase's tables, so that hive can analyse the data stored in hbase
easily.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message