hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hiller, Dean (Contractor)" <dean.hil...@broadridge.com>
Subject IsolationRunner job.jar file not being loaded into classloader?
Date Sun, 02 Jan 2011 22:49:49 GMT
IsolationRunner ends up calling the following code below, and the
"String jar" variable ends up correctly containing the path to job.jar,
and the working Dir(I am in eclipse remote debug mode) contains the
jobxxxx/attemptxxxx/work directory.  There is no lib dir in that work
dir so listFiles returns null below into the libs variable.


But as you can see from the below code, the job.jar is not added to the
URLClassLoader.  I then later get a ClassNotFoundException because
naturally my jar is not in the hadoop classpath and needs to be loaded
like a normal M/R job.  Is there a different location where the jar
should be loaded.  I can obviously cheat and stick my jar in the hadoop
classpath temporarily but that is really not the way to go here....(any
ideas?)  (I can also copy IsolationRunner.java and hack it but would
rather work with what is already there or supply a quick fix as it
doesn't look hard to fix).


  private static ClassLoader makeClassLoader(JobConf conf, 

                                             File workDir) throws
IOException {

    List<URL> cp = new ArrayList<URL>();


    String jar = conf.getJar();

    if (jar != null) {                      // if jar exists, it into

      File[] libs = new File(workDir, "lib").listFiles();

      if (libs != null) {

        for (int i = 0; i < libs.length; i++) {

          cp.add(new URL("file:" + libs[i].toString()));



      cp.add(new URL("file:" + new File(workDir,

      cp.add(new URL("file:" + workDir.toString() + "/"));


    return new URLClassLoader(cp.toArray(new URL[cp.size()]));




This message and any attachments are intended only for the use of the addressee and
may contain information that is privileged and confidential. If the reader of the 
message is not the intended recipient or an authorized representative of the
intended recipient, you are hereby notified that any dissemination of this
communication is strictly prohibited. If you have received this communication in
error, please notify us immediately by e-mail and delete the message and any
attachments from your system.

View raw message