hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack@archive.org (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-189) Add job jar lib, classes, etc. to CLASSPATH when in standalone mode
Date Wed, 03 May 2006 02:13:51 GMT
    [ http://issues.apache.org/jira/browse/HADOOP-189?page=comments#action_12377506 ] 

stack@archive.org commented on HADOOP-189:

The patch seems to have no effect.  When my map goes to run, unless the classes I want to
instantiate are in the root of the job jar, they are not found (ClassNotFoundException). 
Nothing under the job jar /lib nor /classes directory is ever found.  

Tracing, I see that any new class will have null for ClassLoader -- i.e. the "bootstrap class
loader" -- never the URLClassLoader created by this patch. 

Seems like this patch runs too late in the game to be of use?   In order to create a job,
I need to be able to refer to the implementing class (and possibly configurations) BEFORE
LocalJobRunner has had a chance to do its job jar unrolling machinations.  To be able to refer
to implementing classes, they need to be on the immediate CLASSPATH.  

Seems like any messings w/ classloading populating the CLASSPATH needs to happen before we
run Main-Class. This would be most useful I'd imagine.  Would a java program that did nothing
but read hadoop config. for hadoop local and system dir and then unrolled the job jar finishing
by emitting Main-Class and CLASSPATH additions for the hadoop script to exec online #156 be
too ugly?  It would have the advantage of resemblng how job jars are unrolled out on slaves.

I can work up such a patch if amenable.

> Add job jar lib, classes, etc. to CLASSPATH when in standalone mode
> -------------------------------------------------------------------
>          Key: HADOOP-189
>          URL: http://issues.apache.org/jira/browse/HADOOP-189
>      Project: Hadoop
>         Type: New Feature

>   Components: mapred
>     Reporter: stack@archive.org
>  Attachments: local-jar.patch
> Currently, in standalone mode,  hadoop is unable to launch other than the most basic
of job jars where 'basic' is a job jar with nought but class files at top level of the jar
with Main-Class pointing at entry point.  If the job jar has dependencies on jars under the
job jar lib or there are job jar plugins in the classes dir, etc.,  these dependencies are
not loaded and the job fails launch.

This message is automatically generated by JIRA.
If you think it was sent incorrectly contact one of the administrators:
For more information on JIRA, see:

View raw message