hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Loughran <ste...@hortonworks.com>
Subject Re: Lib conflicts
Date Thu, 04 Oct 2012 08:56:12 GMT
On 3 October 2012 12:21, Ben Rycroft <brycroft89@gmail.com> wrote:

> Hi all,
> I have a jar that uses the Hadoop API to launch various remote mapreduce
> jobs (ie, im not using the command-line to initiate the job). The service
> jar that executes the various jobs is built with maven's
> "jar-with-dependencies".
> My jobs all run fine except one that uses commons-codec 1.7, I get:
> FATAL org.apache.hadoop.mapred.Child: Error running child :
> java.lang.NoSuchMethodError:
> org.apache.commons.codec.binary.Base64.encodeAsString([B)Ljava/lang/String;
> I think this is because my jar is including commons-codec 1.7 whereas my
> Hadoop install's lib has commons-codec 1.4 ...
> Is their any way to instruct Hadoop to use the distributed commons-codec
> 1.7 (I assume this is distributed as a job dependency) rather than the
> commons-codec 1.4 in the hadoop 1.0.3 core lib?
Maven's urge to do recursive resolution doesn't help. Play with exclusion
to force the 1.4 out.

FWIW Ivy can be configured to fail if there is any version conflict down
the tree, rather than apply a policy such as "latest wins" or "closest to
root wins"; I don't know if m2 has a similar option

View raw message