hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Joseph Evans (JIRA)" <j...@apache.org>
Subject [jira] [Created] (MAPREDUCE-4740) only .jars can be added to the Distributed Cache classpath
Date Mon, 22 Oct 2012 14:58:13 GMT
Robert Joseph Evans created MAPREDUCE-4740:
----------------------------------------------

             Summary: only .jars can be added to the Distributed Cache classpath
                 Key: MAPREDUCE-4740
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4740
             Project: Hadoop Map/Reduce
          Issue Type: Bug
          Components: mrv2
    Affects Versions: 2.0.2-alpha, 0.23.3
            Reporter: Robert Joseph Evans
            Assignee: Robert Joseph Evans
            Priority: Blocker


Koji tracked down this one.


{noformat}
$ cat mycat.sh
#!/bin/sh
cat >& /dev/null
$JAVA_HOME/bin/jinfo $PPID | grep java.class.path
export | grep CLASSPATH
ls -l 

$ myfile=/user/me/myclasses.zip; yarn jar
hadoop-streaming.jar
-Dmapreduce.job.cache.archives=hdfs:///${myfile}
-Dmapreduce.job.classpath.archives=${myfile} -input in.txt -output out -reducer NONE -mapper
mycat.sh  -file mycat.sh
{noformat}

So, cachearchive like class.zip or class.tar.gz were never set as part of the
classpath even though they were properly set by DistributedCache.addArchiveToClassPath.

It looks like we are parsing the classpath out of the configs, but then throwing that away.
 It looks simple enough to add them in the correct place.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message