hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nitin Pawar <nitinpawar...@gmail.com>
Subject Re: Hive and elasticsearch-hadoop-2.1.0 : NoClassDefFoundError
Date Mon, 29 Jun 2015 14:02:23 GMT
great it helped

On Mon, Jun 29, 2015 at 7:29 PM, Erwan Queffélec <erwan.queffelec@gmail.com>
wrote:

> [continued]
> A dependency for a custom UDF seems not to be properly shaded, as I could
> see in an excerpt of the maven build output:
> [INFO] Including org.apache.httpcomponents:httpclient:jar:4.1.2 in the
> shaded jar.
> [INFO] Including org.apache.httpcomponents:httpcore:jar:4.1.2 in the
> shaded jar.
>
> I'm going to look into this. Thanks a lot for confirming things worked as
> I expected on your end!
>
> Regards,
>
> Erwan
>
> On Mon, Jun 29, 2015 at 3:55 PM, Erwan Queffélec <
> erwan.queffelec@gmail.com> wrote:
>
>> Hi Nitin,
>>
>> Digging up a bit I discovered that the error is probably on our end :
>>
>>
>>
>> On Mon, Jun 29, 2015 at 3:54 PM, Nitin Pawar <nitinpawar432@gmail.com>
>> wrote:
>>
>>> I am using 2.2.4-2.2 but did not get any error.
>>>
>>> can you check what all services are installed on the node where hive
>>> client is running
>>>
>>> On Mon, Jun 29, 2015 at 7:18 PM, Erwan Queffélec <
>>> erwan.queffelec@gmail.com> wrote:
>>>
>>>> Hi Nitin,
>>>>
>>>> No, I didn't do such a thing. I'm using the stock 0.14 version from HDP
>>>> 2.2.4 (said 2.2.6 earlier but that was wrong)
>>>>
>>>> # hive --version
>>>> Hive 0.14.0.2.2.4.2-2
>>>> Subversion
>>>> git://ip-10-0-0-5.ec2.internal/grid/0/jenkins/workspace/HDP-2.2.4.1-centos6/bigtop/build/hive/rpm/BUILD/hive-0.14.0.2.2.4.2
>>>> -r 115d99896f5a4a81e7d91e052e8d38d7436b78d4
>>>> Compiled by jenkins on Tue Mar 31 16:26:33 EDT 2015
>>>> From source with checksum 1f34a1d4e566c3e801582862ed85ee93
>>>>
>>>> Thanks for taking the time.
>>>>
>>>> Kind regards,
>>>>
>>>> Erwan
>>>>
>>>> On Mon, Jun 29, 2015 at 3:44 PM, Nitin Pawar <nitinpawar432@gmail.com>
>>>> wrote:
>>>>
>>>>> by any chance you built hive yourself  ?
>>>>>
>>>>> On Mon, Jun 29, 2015 at 7:11 PM, Erwan Queffélec <
>>>>> erwan.queffelec@gmail.com> wrote:
>>>>>
>>>>>> Additional info : it works when I manually add the jar with ADD JAR
>>>>>> <file>;
>>>>>>
>>>>>> hive> ADD JAR '
>>>>>> /usr/hdp/current/hive-server2/lib/commons-httpclient-3.0.1.jar'
>>>>>>
>>>>>> I'm quite new to hive and hadoop in general. This is my first post
on
>>>>>> this mailing list, so please excuse me if the folowing question has
been
>>>>>> asked & answered over and over again :
>>>>>>
>>>>>> Perhaps I'm a bit naíve, but I though that hive Custom SerDe/UD*Fs
>>>>>> were able to access everything already on the Hive classpath. Was
it just a
>>>>>> dream ?
>>>>>>
>>>>>> I would greatly appreciate some pointers, thanks to anyone who might
>>>>>> be able to help !
>>>>>>
>>>>>> Best regards,
>>>>>>
>>>>>> Erwan
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Mon, Jun 29, 2015 at 2:30 PM, Erwan Queffélec <
>>>>>> erwan.queffelec@gmail.com> wrote:
>>>>>>
>>>>>>> Hello,
>>>>>>>
>>>>>>> I'm running HortonWorks HDP 2.2.6, hive 0.14 alongside an
>>>>>>> ElasticSearch cluster
>>>>>>>
>>>>>>> For some reason Hive can't seem to connect to my ES cluster Using
>>>>>>> the ES SerDe (I'm using elasticsearch-hadoop-2.1.0.jar). When
time comes
>>>>>>> for my job to insert the my query result in an ES resources,
I get this
>>>>>>> stacktrace:
>>>>>>>
>>>>>>> NoClassDefFoundError: org/apache/commons/httpclient/URIException
>>>>>>>  at
>>>>>>> org.elasticsearch.hadoop.hive.HiveUtils.structObjectInspector(HiveUtils.java:57)
>>>>>>>  at org.elasticsearch.hadoop.hive.EsSerDe.initialize(EsSerDe.java:82)
>>>>>>>  at org.elasticsearch.hadoop.hive.EsSerDe.initialize(EsSerDe.java:97)
>>>>>>>
>>>>>>> I'm quite puzzled because commons-httpclient is supposed to be
on
>>>>>>> the hive-client classpath :
>>>>>>>
>>>>>>> # ls -l
>>>>>>> /usr/hdp/current/hive-server2/lib/commons-httpclient-3.0.1.jar
>>>>>>> -rw-r--r-- 1 root root 279781 Mar 31 20:26
>>>>>>> /usr/hdp/current/hive-server2/lib/commons-httpclient-3.0.1.jar
>>>>>>> # ls -l /usr/hdp/current/hive-client/lib/commons-httpclient-3.0.1.jar
>>>>>>> -rw-r--r-- 1 root root 279781 Mar 31 20:26
>>>>>>> /usr/hdp/current/hive-client/lib/commons-httpclient-3.0.1.jar
>>>>>>>
>>>>>>> What am I missing ?
>>>>>>>
>>>>>>> Thanks a lot for your help,
>>>>>>>
>>>>>>> Kind regards,
>>>>>>>
>>>>>>> Erwan
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Nitin Pawar
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Nitin Pawar
>>>
>>
>>
>


-- 
Nitin Pawar

Mime
View raw message