Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of harsh@cloudera.com designates
 209.85.214.176 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAJPnWdReFw422mbh8fyQPotURY-9oeDTo34qCSv-QTQqy8pLbA@mail.gmail.com>
References: 
 <CAJPnWdReFw422mbh8fyQPotURY-9oeDTo34qCSv-QTQqy8pLbA@mail.gmail.com>
From: Harsh J <harsh@cloudera.com>
Date: Wed, 3 Oct 2012 17:09:03 +0530
Message-ID: 
 <CAOcnVr32AwBgdKA_fk4QJSgEd0f272zpUguCHeQxZorWbvUmyg@mail.gmail.com>
Subject: Re: Lib conflicts
To: user@hadoop.apache.org
Content-Type: text/plain; charset=ISO-8859-1

Hi Ben,

As long as the switch of libraries doesn't impact the execution of the
Child task code itself, for Apache Hadoop 1.x, using the config
"mapreduce.user.classpath.first" set to true may solve your trouble.

On Wed, Oct 3, 2012 at 4:51 PM, Ben Rycroft <brycroft89@gmail.com> wrote:
> Hi all,
>
> I have a jar that uses the Hadoop API to launch various remote mapreduce
> jobs (ie, im not using the command-line to initiate the job). The service
> jar that executes the various jobs is built with maven's
> "jar-with-dependencies".
>
> My jobs all run fine except one that uses commons-codec 1.7, I get:
>
> FATAL org.apache.hadoop.mapred.Child: Error running child :
> java.lang.NoSuchMethodError:
> org.apache.commons.codec.binary.Base64.encodeAsString([B)Ljava/lang/String;
>
> I think this is because my jar is including commons-codec 1.7 whereas my
> Hadoop install's lib has commons-codec 1.4 ...
>
> Is their any way to instruct Hadoop to use the distributed commons-codec 1.7
> (I assume this is distributed as a job dependency) rather than the
> commons-codec 1.4 in the hadoop 1.0.3 core lib?
>
> Removing commons-codec-1.4.jar from my Hadoop library folder did seem to
> solve the problem for a bit, but is not working on another VM. Replacing the
> 1.4 jar with the 1.7 does seem to fix the problem but this doesn't seem too
> sane. Hopefully there is a better alternative.
>
> Thanks!


-- 
Harsh J