hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Armstrong <john.armstr...@ccri.com>
Subject Re: Problems adding JARs to distributed classpath in Hadoop 0.20.2
Date Tue, 31 May 2011 19:09:28 GMT
On Tue, 31 May 2011 12:02:28 -0700, Alejandro Abdelnur <tucu@cloudera.com>
wrote:
> What is exactly that does not work?

Oozie launches a wrapper MapReduce job to run a Java job J1.  Oozie's
/lib/ directory is provided to the classpath of J1 as expected.  This part
works.

The Java job J1 configures and launches a MapReduce job MR1.  As part of
the configuration, J1 tries to put some JARs on the distributed classpath
for MR1 to use in its mappers and reducers.  To do so, it calls
DistributedCache.addFileToClassPath(jarfilePath).  The file at jarfilePath
DOES get added to the distributed cache.  But the mapper for MR1 still
throws a ClassNotFoundException, since the file at jarfilePath is NOT on
the classpath for MR1.  This is what doesn't work.

I hope this explanation makes more sense.  Thanks again for putting some
thought to it.

Mime
View raw message