hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Sichi <jsi...@facebook.com>
Subject Re: Hive-Hbase integration problem, ask for help
Date Thu, 17 Jun 2010 20:11:32 GMT
I've added this on as extra validation which ought to be added in HIVE-1222.

JVS

On Jun 15, 2010, at 3:59 PM, Basab Maulik wrote:

> I was not able to reproduce this problem on trunk (can't remember the
> label). The funny thing was both the create table and the insert overwrite
> worked even though the create table contained the invalid row format spec.
> 
> Basab
> 
> On Fri, Jun 11, 2010 at 1:33 PM, John Sichi <jsichi@facebook.com> wrote:
> 
>> You should not be specifying any ROW FORMAT for an HBase table.
>> 
>> From the log in your earlier post, I couldn't tell what was going wrong; I
>> don' think it contained the full exception stacks.  You might be able to dig
>> around in the actual log files to find more.
>> 
>> JVS
>> ________________________________________
>> From: Zhou Shuaifeng [zhoushuaifeng@huawei.com]
>> Sent: Thursday, June 10, 2010 7:26 PM
>> To: hive-dev@hadoop.apache.org
>> Cc: 'zhaozhifeng 00129982'
>> Subject: Hive-Hbase integration problem, ask for help
>> 
>> Hi Guys,
>> 
>> I download the hive source from SVN server, build it and try to run the
>> hive-hbase integration.
>> 
>> I works well on all file-based hive tables, but on the hbase-based tables,
>> the 'insert' command cann't run successful. The 'select' command can run
>> well.
>> 
>> error info is below:
>> 
>> hive> INSERT OVERWRITE TABLE hive_zsf SELECT * FROM zsf WHERE id=3;
>> Total MapReduce jobs = 1
>> Launching Job 1 out of 1
>> Number of reduce tasks is set to 0 since there's no reduce operator
>> Starting Job = job_201006081948_0021, Tracking URL =
>> http://linux-01:50030/jobdetails.jsp?jobid=job_201006081948_0021
>> Kill Command = /opt/hadoop/hdfs/bin/../bin/hadoop job
>> -Dmapred.job.tracker=linux-01:9001 -kill job_201006081948_0021
>> 2010-06-09 16:05:43,898 Stage-0 map = 0%,  reduce = 0%
>> 2010-06-09 16:06:12,131 Stage-0 map = 100%,  reduce = 100%
>> Ended Job = job_201006081948_0021 with errors
>> 
>> Task with the most failures(4):
>> -----
>> Task ID:
>> task_201006081948_0021_m_000000
>> 
>> URL:
>> http://linux-01:50030/taskdetails.jsp?jobid=job_201006081948_0021
>> <
>> http://linux-01:50030/taskdetails.jsp?jobid=job_201006081948_0021&tipid=tas
>> k_201006081948_0021_m_000000<http://linux-01:50030/taskdetails.jsp?jobid=job_201006081948_0021&tipid=tas%0Ak_201006081948_0021_m_000000>>
>> &tipid=task_201006081948_0021_m_000000
>> -----
>> 
>> FAILED: Execution Error, return code 2 from
>> org.apache.hadoop.hive.ql.exec.ExecDriver
>> 
>> 
>> 
>> 
>> I create a hbase-based table with hive, put some data into the hbase table
>> through the hbase shell, and can select data from it through hive:
>> 
>> CREATE TABLE hive_zsf1(id int, name string) ROW FORMAT DELIMITED
>> FIELDS TERMINATED BY '\t'
>> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
>> WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val")
>> TBLPROPERTIES ("hbase.table.name" = "hive_zsf1");
>> 
>> hbase(main):001:0> scan 'hive_zsf1'
>> ROW                          COLUMN+CELL
>> 
>> 1                           column=cf1:val, timestamp=1276157509028,
>> value=zsf
>> 2                           column=cf1:val, timestamp=1276157539051,
>> value=zzf
>> 3                           column=cf1:val, timestamp=1276157548247,
>> value=zw
>> 4                           column=cf1:val, timestamp=1276157557115,
>> value=cjl
>> 4 row(s) in 0.0470 seconds
>> hbase(main):002:0>
>> 
>> hive> select * from hive_zsf1 where id=3;
>> Total MapReduce jobs = 1
>> Launching Job 1 out of 1
>> Number of reduce tasks is set to 0 since there's no reduce operator
>> Starting Job = job_201006081948_0038, Tracking URL =
>> http://linux-01:50030/jobdetails.jsp?jobid=job_201006081948_0038
>> Kill Command = /opt/hadoop/hdfs/bin/../bin/hadoop job
>> -Dmapred.job.tracker=linux-01:9001 -kill job_201006081948_0038
>> 2010-06-11 10:25:42,049 Stage-1 map = 0%,  reduce = 0%
>> 2010-06-11 10:25:45,090 Stage-1 map = 100%,  reduce = 0%
>> 2010-06-11 10:25:48,133 Stage-1 map = 100%,  reduce = 100%
>> Ended Job = job_201006081948_0038
>> OK
>> 3       zw
>> Time taken: 13.526 seconds
>> hive>
>> 
>> 
>> 
>> 
>> 
>> ----------------------------------------------------------------------------
>> ---------------------------------------------------------
>> This e-mail and its attachments contain confidential information from
>> HUAWEI, which
>> is intended only for the person or entity whose address is listed above.
>> Any
>> use of the
>> information contained herein in any way (including, but not limited to,
>> total or partial
>> disclosure, reproduction, or dissemination) by persons other than the
>> intended
>> recipient(s) is prohibited. If you receive this e-mail in error, please
>> notify the sender by
>> phone or email immediately and delete it!
>> 
>> 
>> 


Mime
View raw message