Mailing-List: contact hive-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hive-dev@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of jsichi@facebook.com designates
 69.63.184.103 as permitted sender)
From: John Sichi <jsichi@facebook.com>
To: "hive-dev@hadoop.apache.org" <hive-dev@hadoop.apache.org>
CC: zhaozhifeng 00129982 <zhzf1511@huawei.com>, "zhoushuaifeng@huawei.com"
	<zhoushuaifeng@huawei.com>
Subject: Re: Hive-Hbase integration problem, ask for help
Thread-Topic: Hive-Hbase integration problem, ask for help
Thread-Index: AcsM3oIrerxGyMCMS5Oi7uVPD7PE9ABtXFsA
Date: Thu, 17 Jun 2010 20:11:32 +0000
Message-ID: <F1A0DD32-8693-4E0A-A0CC-3FBC4F55D39C@facebook.com>
References: <000a01cb090d$8e1d2290$26250102@china.huawei.com>
 <3120E6F5005EE7419C125CE166D55E9007770D991D@SC-MBXC1.TheFacebook.com>
 <AANLkTimL4XiOO9exPrFsN5X2Q3-8sl4-5PmLEL-zUEjU@mail.gmail.com>
In-Reply-To: <AANLkTimL4XiOO9exPrFsN5X2Q3-8sl4-5PmLEL-zUEjU@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
Content-Type: text/plain; charset="us-ascii"
Content-ID: <039f6a86-26be-4727-8bf2-2dc2178738e0>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

I've added this on as extra validation which ought to be added in HIVE-1222=
.

JVS

On Jun 15, 2010, at 3:59 PM, Basab Maulik wrote:

> I was not able to reproduce this problem on trunk (can't remember the
> label). The funny thing was both the create table and the insert overwrit=
e
> worked even though the create table contained the invalid row format spec=
.
>=20
> Basab
>=20
> On Fri, Jun 11, 2010 at 1:33 PM, John Sichi <jsichi@facebook.com> wrote:
>=20
>> You should not be specifying any ROW FORMAT for an HBase table.
>>=20
>> From the log in your earlier post, I couldn't tell what was going wrong;=
 I
>> don' think it contained the full exception stacks.  You might be able to=
 dig
>> around in the actual log files to find more.
>>=20
>> JVS
>> ________________________________________
>> From: Zhou Shuaifeng [zhoushuaifeng@huawei.com]
>> Sent: Thursday, June 10, 2010 7:26 PM
>> To: hive-dev@hadoop.apache.org
>> Cc: 'zhaozhifeng 00129982'
>> Subject: Hive-Hbase integration problem, ask for help
>>=20
>> Hi Guys,
>>=20
>> I download the hive source from SVN server, build it and try to run the
>> hive-hbase integration.
>>=20
>> I works well on all file-based hive tables, but on the hbase-based table=
s,
>> the 'insert' command cann't run successful. The 'select' command can run
>> well.
>>=20
>> error info is below:
>>=20
>> hive> INSERT OVERWRITE TABLE hive_zsf SELECT * FROM zsf WHERE id=3D3;
>> Total MapReduce jobs =3D 1
>> Launching Job 1 out of 1
>> Number of reduce tasks is set to 0 since there's no reduce operator
>> Starting Job =3D job_201006081948_0021, Tracking URL =3D
>> http://linux-01:50030/jobdetails.jsp?jobid=3Djob_201006081948_0021
>> Kill Command =3D /opt/hadoop/hdfs/bin/../bin/hadoop job
>> -Dmapred.job.tracker=3Dlinux-01:9001 -kill job_201006081948_0021
>> 2010-06-09 16:05:43,898 Stage-0 map =3D 0%,  reduce =3D 0%
>> 2010-06-09 16:06:12,131 Stage-0 map =3D 100%,  reduce =3D 100%
>> Ended Job =3D job_201006081948_0021 with errors
>>=20
>> Task with the most failures(4):
>> -----
>> Task ID:
>> task_201006081948_0021_m_000000
>>=20
>> URL:
>> http://linux-01:50030/taskdetails.jsp?jobid=3Djob_201006081948_0021
>> <
>> http://linux-01:50030/taskdetails.jsp?jobid=3Djob_201006081948_0021&tipi=
d=3Dtas
>> k_201006081948_0021_m_000000<http://linux-01:50030/taskdetails.jsp?jobid=
=3Djob_201006081948_0021&tipid=3Dtas%0Ak_201006081948_0021_m_000000>>
>> &tipid=3Dtask_201006081948_0021_m_000000
>> -----
>>=20
>> FAILED: Execution Error, return code 2 from
>> org.apache.hadoop.hive.ql.exec.ExecDriver
>>=20
>>=20
>>=20
>>=20
>> I create a hbase-based table with hive, put some data into the hbase tab=
le
>> through the hbase shell, and can select data from it through hive:
>>=20
>> CREATE TABLE hive_zsf1(id int, name string) ROW FORMAT DELIMITED
>> FIELDS TERMINATED BY '\t'
>> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
>> WITH SERDEPROPERTIES ("hbase.columns.mapping" =3D ":key,cf1:val")
>> TBLPROPERTIES ("hbase.table.name" =3D "hive_zsf1");
>>=20
>> hbase(main):001:0> scan 'hive_zsf1'
>> ROW                          COLUMN+CELL
>>=20
>> 1                           column=3Dcf1:val, timestamp=3D1276157509028,
>> value=3Dzsf
>> 2                           column=3Dcf1:val, timestamp=3D1276157539051,
>> value=3Dzzf
>> 3                           column=3Dcf1:val, timestamp=3D1276157548247,
>> value=3Dzw
>> 4                           column=3Dcf1:val, timestamp=3D1276157557115,
>> value=3Dcjl
>> 4 row(s) in 0.0470 seconds
>> hbase(main):002:0>
>>=20
>> hive> select * from hive_zsf1 where id=3D3;
>> Total MapReduce jobs =3D 1
>> Launching Job 1 out of 1
>> Number of reduce tasks is set to 0 since there's no reduce operator
>> Starting Job =3D job_201006081948_0038, Tracking URL =3D
>> http://linux-01:50030/jobdetails.jsp?jobid=3Djob_201006081948_0038
>> Kill Command =3D /opt/hadoop/hdfs/bin/../bin/hadoop job
>> -Dmapred.job.tracker=3Dlinux-01:9001 -kill job_201006081948_0038
>> 2010-06-11 10:25:42,049 Stage-1 map =3D 0%,  reduce =3D 0%
>> 2010-06-11 10:25:45,090 Stage-1 map =3D 100%,  reduce =3D 0%
>> 2010-06-11 10:25:48,133 Stage-1 map =3D 100%,  reduce =3D 100%
>> Ended Job =3D job_201006081948_0038
>> OK
>> 3       zw
>> Time taken: 13.526 seconds
>> hive>
>>=20
>>=20
>>=20
>>=20
>>=20
>> ------------------------------------------------------------------------=
----
>> ---------------------------------------------------------
>> This e-mail and its attachments contain confidential information from
>> HUAWEI, which
>> is intended only for the person or entity whose address is listed above.
>> Any
>> use of the
>> information contained herein in any way (including, but not limited to,
>> total or partial
>> disclosure, reproduction, or dissemination) by persons other than the
>> intended
>> recipient(s) is prohibited. If you receive this e-mail in error, please
>> notify the sender by
>> phone or email immediately and delete it!
>>=20
>>=20
>>=20