hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Benjamin Reed (JIRA)" <j...@apache.org>
Subject [jira] Created: (HADOOP-303) JobClient looking for classes for submitted job in the wrong place
Date Wed, 14 Jun 2006 19:07:29 GMT
JobClient looking for classes for submitted job in the wrong place

         Key: HADOOP-303
         URL: http://issues.apache.org/jira/browse/HADOOP-303
     Project: Hadoop
        Type: Bug

  Components: mapred  
    Versions: 0.3.2    
    Reporter: Benjamin Reed
    Priority: Minor

JobClient does some checking of the job being submitted when it submits a jar file along with
the job. The problem is that the JobClient pulls classes from the classpath rather than the
submitted jar file. Because the jar file may contain newer (or older) versions of classes
on the classpath this behavior leads to confusing errors when the job is run. It is also a
pain to ensure that the jar file being submitted is on the classpath. Further, if the JobClient
uses the submitted jar file rather than the classpath, missing classes from the jar file can
be detected earlier.

This patch will cause the JobClient to load the classes for a job from the jar file rather
than the classpath. Because of the class loading precedence rules in Java, if the class is
on the system class path, it will be loaded from there rather than the submitted jar file,
but now users need not (and should not) put job classfiles on the system classpath.

This patch also allows config files to be put in a configuration directory rather than on
the classpath, which also eliminates some confusing behavior when there are duplicate instances
of config files in different parts of the classpath. 

This message is automatically generated by JIRA.
If you think it was sent incorrectly contact one of the administrators:
For more information on JIRA, see:

View raw message