hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Liren Ding <sky.gonna.bri...@gmail.com>
Subject Re: Hadoop library version issues and conflict
Date Thu, 28 Jan 2016 21:54:07 GMT
I saw a few posts mentioned the property "mapreduce.job.user.classpath.first"
could be used to handle this situation. But it seems to have been
deprecated in yarn. Does anyone know any alternative?


On Thu, Jan 28, 2016 at 12:19 PM, Solomon Duskis <sduskis@gmail.com> wrote:

> I'm not sure about the solution here, but it would be nice to have much
> newer dependencies of everything.  There are a host of issues with newer
> libraries running in the same VM as hbase and hadoop.  Even the newer hbase
> libraries are 3+ year old.
>
> On Thu, Jan 28, 2016 at 2:33 PM, Liren Ding <sky.gonna.bright@gmail.com>
> wrote:
>
>> My MR job use following libs:
>>
>>    - commons-codec-1.9,
>>    - commons-compress-1.7,
>>    - guava-12.0.1,
>>    - httpclient-4.3.1
>>    - httpcore-4.
>>
>> But the versions in hadoop libs are:
>>
>>    - commons-codec-1.4
>>    - commons-compress-1.4.1
>>    - guava-11.0.2
>>    - httpclient-4.2.5
>>    - httpcore-4.2.5
>>
>> This conflict causes a "java.lang.NoSuchMethodError" error. One insane
>> solution is replacing all libs allover the cluster with new versions. But
>> the approach is not practical in production environment. Is there a decent
>> way to solve the hadoop libraries version conflict issue?
>>
>
>

Mime
View raw message