Return-Path: X-Original-To: apmail-hive-user-archive@www.apache.org Delivered-To: apmail-hive-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id CCFF2CDAB for ; Fri, 14 Mar 2014 18:30:46 +0000 (UTC) Received: (qmail 92200 invoked by uid 500); 14 Mar 2014 18:30:37 -0000 Delivered-To: apmail-hive-user-archive@hive.apache.org Received: (qmail 91901 invoked by uid 500); 14 Mar 2014 18:30:36 -0000 Mailing-List: contact user-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hive.apache.org Delivered-To: mailing list user@hive.apache.org Received: (qmail 91890 invoked by uid 500); 14 Mar 2014 18:30:34 -0000 Delivered-To: apmail-hadoop-hive-user@hadoop.apache.org Received: (qmail 91883 invoked by uid 99); 14 Mar 2014 18:30:34 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 14 Mar 2014 18:30:34 +0000 X-ASF-Spam-Status: No, hits=2.5 required=5.0 tests=FREEMAIL_REPLY,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of ndimiduk@gmail.com designates 209.85.220.182 as permitted sender) Received: from [209.85.220.182] (HELO mail-vc0-f182.google.com) (209.85.220.182) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 14 Mar 2014 18:30:30 +0000 Received: by mail-vc0-f182.google.com with SMTP id ks9so3132850vcb.41 for ; Fri, 14 Mar 2014 11:30:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=DcX2xhWOxrbB+4qgBvAVm0adF/mfHVvmFfBvwyylgVM=; b=FtNLiuon985mJmu8dQBipvlI4YcnHmtnwKCe3zfka2I0u4Iemk5GjO4QGQk8uh1WQn +Beio3m1APM6PK18BV00JYm2hyN6bwR1xP4PbAHahD2DK66t7BNbJ/UQxuT7Dwx+aZQ9 10qWqYHYJbE5o4oZT0WXxeaZxdl9LGYrjSW6g6CyoZJrmPf5yx8QxC1avELMVHr4jwOK Mkpo6VjMthXXoRqyvx2EHxBT0rRM59j15vz7iTzrvUvcygwhctR9lD8vPo8JVErrecRQ LYEOs7ZB5fuwhrHnlX2hUHh2UYp3JvSZwb8mat2QhqzPonMeHtzYo3g1YqC+0NTunGQ1 bahw== X-Received: by 10.52.12.36 with SMTP id v4mr6402281vdb.20.1394821809997; Fri, 14 Mar 2014 11:30:09 -0700 (PDT) MIME-Version: 1.0 Received: by 10.58.12.232 with HTTP; Fri, 14 Mar 2014 11:29:49 -0700 (PDT) In-Reply-To: References: From: Nick Dimiduk Date: Fri, 14 Mar 2014 11:29:49 -0700 Message-ID: Subject: Re: HBase- Hive Integration To: hbase-user , hive-user@hadoop.apache.org Cc: Rekha_Joshi@intuit.com, kulkarni.swarnim@gmail.com Content-Type: multipart/alternative; boundary=485b397dd2998290e804f4954223 X-Virus-Checked: Checked by ClamAV on apache.org --485b397dd2998290e804f4954223 Content-Type: text/plain; charset=UTF-8 hbase-dev to bcc; adding hive-user. this is a question for the user lists, and more for Hive's than HBase, as HBaseStorageHandler is code in Hive project, not HBase. Hi Sai, You are embarking into a brave world. Because your aim is the interop across these different Apache projects, I highly recommend using a vendor's distribution. The reason being that the vendor has done the work of certifying interoperability for you, so you won't be left finding bugs in the edges that aren't well tested in the communities. Notice in the blog post referenced, I was using a distribution and even then I've found interop issues. Hive 0.11 will require you manually specify the HBase jars. Please don't copy them around, instead use environment variables. You can follow along with my blog post and set HADOOP_CLASSPATH and HIVE_AUX_JARS_PATH appropriately. You're using HBase 0.96.1, which includes HBASE-8438, so you can use `hbase mapredcp` instead of itemizing jars manually. This requirement (hopefully) goes away entirely in Hive 0.13 with HIVE-2379 and HIVE-2055. Finally, are you using Hive on the CLI or through a web tool (Hue?). The details change slightly based on all these... details. Good luck. Keep asking questions. Please file bugs. Thanks, Nick On Thu, Mar 13, 2014 at 10:56 PM, Sai Pavan Gadde wrote: > Hi, > > Here is my document in an attached file. It contains my flow of hive-hbase > integration. > 1. I created a table in mysql and imported into hbase, > 2. I copied a hive-handler.jar, guva.jar and zookeeper.jar file in > Hadoop-2.2.0/share/mapred/lib folder > 3. copied required hive(0.11.0) jars to hbase(0.96.1.1-hadoop2) && hbase > jars to hive > 4. Trying to create a table in hive with hbase properties but gets error as > > > *FAILED: Error in metadata: MetaException(message:java.io.IOException: > java.lang.reflect.InvocationTargetException* > > > I think so hbase-0.96.1.1-hadoop2.jar file is not present in HBase > directory. > > So, Suggest for further step to proceed. > > > On Thu, Mar 13, 2014 at 7:12 PM, kulkarni.swarnim@gmail.com < > kulkarni.swarnim@gmail.com> wrote: > >> Hi Sai, >> >> If you can show us what specific exception you are seeing, we would be >> able >> to help you out in a better way. >> >> Thanks, >> >> >> On Thu, Mar 13, 2014 at 7:44 AM, Sai Pavan Gadde > >wrote: >> >> > Hi, >> > Thanks for reply... but >> > >> > I worked with hadoop-1.2.1 which is fine working, but need with real >> > requirement is Hadoop-2.2.0 i.e, YARN >> > >> > Please show the way to get solve with apache products >> > >> > I Installed HBase-0.96.1.1-hadoop2 on top of hadoop-2.2.0, installed >> > hive-0.11.0 too, >> > >> > Copied hive-hbase-handler-0.11.0 to hadoop-2.2.0, guva.jar, >> zookeeper.jar >> > >> > I created hbase-0.96.1.1-hadoop2.jar(which is absent in version) and >> worked >> > but not getting connected with hive. I suppose to work with >> hadoop-0.94.x >> > but the version is not compatible with hadoop-2.2.0 >> > >> > I tried in all the way, not found the solution, please suggest me >> > >> > >> > >> > >> > On Thu, Mar 13, 2014 at 4:20 PM, Joshi, Rekha > > >wrote: >> > >> > > Hi, >> > > >> > > We have successfully created external hive table pointing to hbase >> table >> > > with settings for hive, zookeeper, HBase security tokens. >> > > >> > > If it is a jar problem, then mostly the env is not set correctly.Hive >> > > should be able to recognize HBaseStorageHandler. >> > > >> > > Thanks >> > > Rekha >> > > >> > > http://hortonworks.com/blog/using-hive-to-interact-with-hbase-part-2/ >> > > >> > > >> > > >> > > On 3/13/14, 10:15 AM, "Sai Pavan Gadde" wrote: >> > > >> > > >Hi, >> > > > >> > > >I am using hadoop-2.2.0 , HBASE-0.96.1.1-hadoop2 and hive-0.11.0 >> version >> > > >for a project. I got a requirement such that i have to integrate >> hbase >> > to >> > > >hive , so that table reflection should happens. >> > > > >> > > >while i am doing this, i got error like TARGET INNOVOCATION >> EXCEPTION. >> > > >This is because jar file target missing. please suggest me to >> overcome >> > the >> > > >problem >> > > > >> > > >-- >> > > >*THANKS & REGARDS,* >> > > >G.SAI PAVAN, >> > > >CCDH4 CERTIFIED, >> > > >Ph: 8121914494, >> > > >*www.bigdatatrendz.com * >> > > >linkedin profile < >> > http://in.linkedin.com/pub/gadde-sai-pavan/38/44b/453/> >> > > >HYDERABAD. >> > > >> > > >> > >> > >> > -- >> > *THANKS & REGARDS,* >> > G.SAI PAVAN, >> > CCDH4 CERTIFIED, >> > Ph: 8121914494, >> > *www.bigdatatrendz.com * >> > linkedin profile < >> http://in.linkedin.com/pub/gadde-sai-pavan/38/44b/453/> >> > HYDERABAD. >> > >> >> >> >> -- >> Swarnim >> > > > > -- > *THANKS & REGARDS,* > G.SAI PAVAN, > CCDH4 CERTIFIED, > Ph: 8121914494, > *www.bigdatatrendz.com * > linkedin profile > HYDERABAD. > --485b397dd2998290e804f4954223 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
hbase-dev to bcc; adding hive-user. this is a question for= the user lists, and more for Hive's than HBase, as HBaseStorageHandler= is code in Hive project, not HBase.

Hi Sai,

You are embarking into a brave world. Because your aim is th= e interop across these different Apache projects, I highly recommend using = a vendor's distribution. The reason being that the vendor has done the = work of certifying interoperability for you, so you won't be left findi= ng bugs in the edges that aren't well tested in the communities. Notice= in the blog post referenced, I was using a distribution and even then I= 9;ve found interop issues.

Hive 0.11 will require you manually specify the HBase j= ars. Please don't copy them around, instead use environment variables. = You can follow along with my blog post and set HADOOP_CLASSPATH and HIVE_AU= X_JARS_PATH appropriately. You're using HBase 0.96.1, which includes HB= ASE-8438, so you can use `hbase mapredcp` instead of itemizing jars manuall= y. This requirement (hopefully) goes away entirely in Hive 0.13 with=C2=A0H= IVE-2379 and=C2=A0HIVE-2055.

Finally, are you using Hive on the CLI or through a web= tool (Hue?). The details change slightly based on all these... details.

Good luck. Keep asking questions. Please file bugs.<= /div>

Thanks,
Nick


On Thu, Mar 13, 2014 at 10:56 PM, = Sai Pavan Gadde <id4gpavan@gmail.com> wrote:
Hi,
Here is my document in an attached file. It contains my flow of hive-h= base integration.
1. I created a table in mysql and imported into hbase,
2. I= copied a hive-handler.jar, guva.jar and zookeeper.jar file in Hadoop-2.2.0= /share/mapred/lib folder
3. copied required hive(0.11.0) jars to hbase(0.96.1.1-hadoop2) &= & hbase jars to hive
4. Trying to create a table in hive with = hbase properties but gets error as
=C2=A0=C2=A0=C2=A0=C2=A0=20 =09 =09 =09 =09

FAILED: Error in metadata: MetaException(message:java.io.IOException: java.lang.reflect.InvocationTargetException


I think so hbase-0.96.1.1-had= oop2.jar file is not present in=C2=A0 HBase directory.

So, Suggest for further step to proceed.



On Thu, Mar 13, 2014 at 7:12 PM, kulkarni.swarnim@gmail.com <kulkarni.swarnim@gmail.com> wrote:
Hi Sai,

If you can show us what specific exception you are seeing, we would be able=
to help you out in a better way.

Thanks,


On Thu, Mar 13, 2014 at 7:44 AM, Sai Pavan Gadde <id4gpavan@gmail.com>wrote:

> Hi,
> Thanks for reply... but
>
> I worked with hadoop-1.2.1 which is fine working, but need with real > requirement is Hadoop-2.2.0 i.e, YARN
>
> Please show the way to get solve with apache products
>
> I Installed HBase-0.96.1.1-hadoop2 on top of hadoop-2.2.0, installed > hive-0.11.0 too,
>
> Copied hive-hbase-handler-0.11.0 to hadoop-2.2.0, guva.jar, zookeeper.= jar
>
> I created hbase-0.96.1.1-hadoop2.jar(which is absent in version) and w= orked
> but not getting connected with hive. I suppose to work with hadoop-0.9= 4.x
> but the version is not compatible with hadoop-2.2.0
>
> I tried in all the way, not found the solution, please suggest me
>
>
>
>
> On Thu, Mar 13, 2014 at 4:20 PM, Joshi, Rekha <Rekha_Joshi@intuit.com
> >wrote:
>
> > Hi,
> >
> > We have successfully created external hive table pointing to hbas= e table
> > with settings for hive, zookeeper, HBase security tokens.
> >
> > If it is a jar problem, then mostly the env is not set correctly.= Hive
> > should be able to recognize HBaseStorageHandler.
> >
> > Thanks
> > Rekha
> >
> > http://hortonworks.com/blog/using-hive-t= o-interact-with-hbase-part-2/
> >
> >
> >
> > On 3/13/14, 10:15 AM, "Sai Pavan Gadde" <id4gpavan@gmail.com> w= rote:
> >
> > >Hi,
> > >
> > >I am using hadoop-2.2.0 , HBASE-0.96.1.1-hadoop2 and hive-0.1= 1.0 version
> > >for a project. I got a requirement such that i have to integr= ate hbase
> to
> > >hive , so that table reflection should happens.
> > >
> > >while =C2=A0i am doing this, i got error like TARGET INNOVOCA= TION EXCEPTION.
> > >This is because jar file target missing. please suggest me to= overcome
> the
> > >problem
> > >
> > >--
> > >*THANKS & REGARDS,*
> > >G.SAI PAVAN,
> > >CCDH4 CERTIFIED,
> > >Ph: 8121914494,
> > >*w= ww.bigdatatrendz.com <http://www.bigdatatrendz.com>*
> > >linkedin profile <
> http://in.linkedin.com/pub/gadde-sai-pavan/38/44b/453/&g= t;
> > >HYDERABAD.
> >
> >
>
>
> --
> *THANKS & REGARDS,*
> G.SAI PAVAN,
> CCDH4 CERTIFIED,
> Ph: 8121914494,
> *www.bigdat= atrendz.com <http://www.bigdatatrendz.com>*
> linkedin profile <http://in.linkedin.com/pub/gadde-sai-pa= van/38/44b/453/>
> HYDERABAD.
>



--
Swarnim



--
THANKS = & REGARDS,
G.SAI PAVAN,
CCDH4 CERTIFIED,
=
Ph: 8= 121914494,

--485b397dd2998290e804f4954223--