hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alejandro Abdelnur <t...@cloudera.com>
Subject Re: Problems adding JARs to distributed classpath in Hadoop 0.20.2
Date Tue, 31 May 2011 19:02:28 GMT
What is exactly that does not work?

Oozie uses DistributeCache as the only mechanism to set classpaths to jobs
and it works fine.

Thanks.

Alejandro

On Mon, May 30, 2011 at 10:22 AM, John Armstrong <john.armstrong@ccri.com>wrote:

> On Mon, 30 May 2011 09:43:14 -0700, Alejandro Abdelnur <tucu@cloudera.com>
> wrote:
> > If you still want to start your MR job from your Java action, then your
> > Java
> > action should do all the setup the MapReduceMain class does before
> starting
> > the MR job (this will ensure delegation tokens and distributed cache is
> > avail to your MR job).
>
> Yes, my Java action is doing the setup work.  In particular, it calls
> DistrributedCache.addfileToClassPath(), which (according to the
> documentation) should be the same as passing it in at the command line with
> -libjars, right?  And yet it doesn't seem to work.
>
> Is this the same as the JIRA issue MAPREDUCE-752?  And if so, does this
> mean that there is no solution (other than a workaround like passing a fat
> JAR) that doesn't involve patching the Hadoop code itself (which I'd have
> to get our client to agree to)?
>

Mime
View raw message