Return-Path: X-Original-To: apmail-hive-user-archive@www.apache.org Delivered-To: apmail-hive-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 0BECC18212 for ; Mon, 29 Jun 2015 13:54:23 +0000 (UTC) Received: (qmail 80415 invoked by uid 500); 29 Jun 2015 13:54:21 -0000 Delivered-To: apmail-hive-user-archive@hive.apache.org Received: (qmail 80346 invoked by uid 500); 29 Jun 2015 13:54:21 -0000 Mailing-List: contact user-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hive.apache.org Delivered-To: mailing list user@hive.apache.org Received: (qmail 80335 invoked by uid 99); 29 Jun 2015 13:54:21 -0000 Received: from Unknown (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 29 Jun 2015 13:54:21 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id CF8B218275B for ; Mon, 29 Jun 2015 13:54:20 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.041 X-Spam-Level: ** X-Spam-Status: No, score=2.041 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, HTML_MESSAGE=3, RCVD_IN_MSPIKE_H2=-1.108, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd3-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-us-west.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id LsdA1fRX0eSR for ; Mon, 29 Jun 2015 13:54:19 +0000 (UTC) Received: from mail-ie0-f181.google.com (mail-ie0-f181.google.com [209.85.223.181]) by mx1-us-west.apache.org (ASF Mail Server at mx1-us-west.apache.org) with ESMTPS id 2CF70211EF for ; Mon, 29 Jun 2015 13:54:19 +0000 (UTC) Received: by iecvh10 with SMTP id vh10so115442556iec.3 for ; Mon, 29 Jun 2015 06:54:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=9qGKn0OwtJr0xAs+dsnGwBYMIqhPk3tfX6LE4BzcGus=; b=MaVE7Txz0tr5mUUHoBr5Im/rl9xrkctNb70Uq8D0c02z0vYbN2oieVYVBozvu3HzqP xAgkoW+Ms9wZXD6w99pFFLOl//0dTNaa7eP9xifWXCsDbSOiG/RCIVicnMfcpOu08JJ+ YDazLOoPhcuP0LO9wAvY346dfOJ8hfd7iwARjlaO7VkgsZ4Pp7bcHRrOyVW3MzQqv3b8 BBSdsiCM1jjtLn9ehnGikig4iECLtm64Ac44ojN/DnyZHMMI4vX3oZQpcHHNb+OPea1V 6BS6YE2bAVN7oVztQxqeGu81xaxNrFKEHfKw6XmJs7B2CFz2R3mFN0s6t9prN3Hbh49+ bBCw== MIME-Version: 1.0 X-Received: by 10.50.13.34 with SMTP id e2mr15445247igc.23.1435586058648; Mon, 29 Jun 2015 06:54:18 -0700 (PDT) Received: by 10.64.116.194 with HTTP; Mon, 29 Jun 2015 06:54:18 -0700 (PDT) In-Reply-To: References: Date: Mon, 29 Jun 2015 19:24:18 +0530 Message-ID: Subject: Re: Hive and elasticsearch-hadoop-2.1.0 : NoClassDefFoundError From: Nitin Pawar To: "user@hive.apache.org" Content-Type: multipart/alternative; boundary=089e01182d04122fcd0519a86dfe --089e01182d04122fcd0519a86dfe Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable I am using 2.2.4-2.2 but did not get any error. can you check what all services are installed on the node where hive client is running On Mon, Jun 29, 2015 at 7:18 PM, Erwan Queff=C3=A9lec wrote: > Hi Nitin, > > No, I didn't do such a thing. I'm using the stock 0.14 version from HDP > 2.2.4 (said 2.2.6 earlier but that was wrong) > > # hive --version > Hive 0.14.0.2.2.4.2-2 > Subversion > git://ip-10-0-0-5.ec2.internal/grid/0/jenkins/workspace/HDP-2.2.4.1-cento= s6/bigtop/build/hive/rpm/BUILD/hive-0.14.0.2.2.4.2 > -r 115d99896f5a4a81e7d91e052e8d38d7436b78d4 > Compiled by jenkins on Tue Mar 31 16:26:33 EDT 2015 > From source with checksum 1f34a1d4e566c3e801582862ed85ee93 > > Thanks for taking the time. > > Kind regards, > > Erwan > > On Mon, Jun 29, 2015 at 3:44 PM, Nitin Pawar > wrote: > >> by any chance you built hive yourself ? >> >> On Mon, Jun 29, 2015 at 7:11 PM, Erwan Queff=C3=A9lec < >> erwan.queffelec@gmail.com> wrote: >> >>> Additional info : it works when I manually add the jar with ADD JAR >>> ; >>> >>> hive> ADD JAR ' >>> /usr/hdp/current/hive-server2/lib/commons-httpclient-3.0.1.jar' >>> >>> I'm quite new to hive and hadoop in general. This is my first post on >>> this mailing list, so please excuse me if the folowing question has bee= n >>> asked & answered over and over again : >>> >>> Perhaps I'm a bit na=C3=ADve, but I though that hive Custom SerDe/UD*Fs= were >>> able to access everything already on the Hive classpath. Was it just a >>> dream ? >>> >>> I would greatly appreciate some pointers, thanks to anyone who might be >>> able to help ! >>> >>> Best regards, >>> >>> Erwan >>> >>> >>> >>> On Mon, Jun 29, 2015 at 2:30 PM, Erwan Queff=C3=A9lec < >>> erwan.queffelec@gmail.com> wrote: >>> >>>> Hello, >>>> >>>> I'm running HortonWorks HDP 2.2.6, hive 0.14 alongside an ElasticSearc= h >>>> cluster >>>> >>>> For some reason Hive can't seem to connect to my ES cluster Using the >>>> ES SerDe (I'm using elasticsearch-hadoop-2.1.0.jar). When time comes f= or my >>>> job to insert the my query result in an ES resources, I get this stack= trace: >>>> >>>> NoClassDefFoundError: org/apache/commons/httpclient/URIException >>>> at >>>> org.elasticsearch.hadoop.hive.HiveUtils.structObjectInspector(HiveUtil= s.java:57) >>>> at org.elasticsearch.hadoop.hive.EsSerDe.initialize(EsSerDe.java:82) >>>> at org.elasticsearch.hadoop.hive.EsSerDe.initialize(EsSerDe.java:97) >>>> >>>> I'm quite puzzled because commons-httpclient is supposed to be on the >>>> hive-client classpath : >>>> >>>> # ls -l /usr/hdp/current/hive-server2/lib/commons-httpclient-3.0.1.jar >>>> -rw-r--r-- 1 root root 279781 Mar 31 20:26 >>>> /usr/hdp/current/hive-server2/lib/commons-httpclient-3.0.1.jar >>>> # ls -l /usr/hdp/current/hive-client/lib/commons-httpclient-3.0.1.jar >>>> -rw-r--r-- 1 root root 279781 Mar 31 20:26 >>>> /usr/hdp/current/hive-client/lib/commons-httpclient-3.0.1.jar >>>> >>>> What am I missing ? >>>> >>>> Thanks a lot for your help, >>>> >>>> Kind regards, >>>> >>>> Erwan >>>> >>> >>> >> >> >> -- >> Nitin Pawar >> > > --=20 Nitin Pawar --089e01182d04122fcd0519a86dfe Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
I am using 2.2.4-2.2 but did not get any error.
<= br>
can you check what all services are installed on the node whe= re hive client is running

On Mon, Jun 29, 2015 at 7:18 PM, Erwan Queff=C3=A9= lec <erwan.queffelec@gmail.com> wrote:
Hi Nitin,

No, I didn't do such a thing. I'm using the stock 0.14 versi= on from HDP 2.2.4 (said 2.2.6 earlier but that was wrong)

# hive --version
Hive 0.14.0.2.2.4.2-2
Subversion git://ip= -10-0-0-5.ec2.internal/grid/0/jenkins/workspace/HDP-2.2.4.1-centos6/bigtop/= build/hive/rpm/BUILD/hive-0.14.0.2.2.4.2 -r 115d99896f5a4a81e7d91e052e8d38d= 7436b78d4
Compiled by jenkins on Tue Mar 31 16:26:33 EDT 2015
From so= urce with checksum 1f34a1d4e566c3e801582862ed85ee93

Tha= nks for taking the time.

Kind regards,

Erwan
=
On Mon, Jun 29, 2015 at 3:44 PM, Nitin Pawar <= span dir=3D"ltr"><nitinpawar432@gmail.com> wrote:
by any chance you built hive yourself=C2=A0 = ?

On Mon, Jun 29, 2015 at 7:11 PM, Erwan Queff=C3=A9lec <erwan.= queffelec@gmail.com> wrote:
Additional info : it works when I manually add = the jar with ADD JAR <file>;

hive> ADD JAR '/usr/hdp/current/hive-server2/lib/commons-httpclient= -3.0.1.jar'

I'm quite new t= o hive and hadoop in general. This is my first post on this mailing list, s= o please excuse me if the folowing question has been asked & answered o= ver and over again :

Perhaps I'm a bit na=C3=ADve, but I though = that hive Custom SerDe/UD*Fs were able to access everything already on the = Hive classpath. Was it just a dream ?

I would greatly appreciate some pointers, thanks to anyone who mi= ght be able to help !

Best r= egards,

Erwan


<= div>

On Mon, Jun 2= 9, 2015 at 2:30 PM, Erwan Queff=C3=A9lec <erwan.queffelec@gmail.co= m> wrote:
=
Hello,

I'm running HortonWorks HDP 2.2.6, hive 0= .14 alongside an ElasticSearch cluster

For some reason Hive ca= n't seem to connect to my ES cluster Using the ES SerDe (I'm using = elasticsearch-hadoop-2.1.0.jar). When time comes for my job to insert the m= y query result in an ES resources, I get this stacktrace:

NoClassDefFoundError: org/apache/commons/httpclient/URIExcep= tion
=C2=A0at or= g.elasticsearch.hadoop.hive.HiveUtils.structObjectInspector(HiveUtils.java:= 57)
=C2=A0at org.= elasticsearch.hadoop.hive.EsSerDe.initialize(EsSerDe.java:82)
=C2=A0at org.elasticsearch.had= oop.hive.EsSerDe.initialize(EsSerDe.java:97)

I'm quite puzzled because commons-httpclient is supp= osed to be on the hive-client classpath :

# ls -l /usr/hdp/curr= ent/hive-server2/lib/commons-httpclient-3.0.1.jar
-rw-r--r-- 1 root root 279781 Mar 31 20:26= /usr/hdp/current/hive-server2/lib/commons-httpclient-3.0.1.jar
<= span style=3D"font-family:monospace,monospace"># ls -l /usr/hdp/current/hiv= e-client/lib/commons-httpclient-3.0.1.jar
-rw-r--r-- 1 root root 279781 Mar 31 20:26 /usr/hd= p/current/hive-client/lib/commons-httpclient-3.0.1.jar

=
What am I missing ?

Thanks a lot for your hel= p,

Kind regards,

Erwan
=




= --
Nitin Pawar




--
Nitin Pawar
--089e01182d04122fcd0519a86dfe--