Mailing-List: contact mapreduce-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: mapreduce-issues@hadoop.apache.org
Date: Tue, 8 Mar 2011 18:40:06 +0000 (UTC)
From: "Tony Valderrama (JIRA)" <jira@apache.org>
To: mapreduce-issues@hadoop.apache.org
Message-ID: 
 <713059234.5712.1299609606375.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] Created: (MAPREDUCE-2370) JobConf.findContainingJar
 incorrectly transforms paths containing '+' character
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

JobConf.findContainingJar incorrectly transforms paths containing '+' character
-------------------------------------------------------------------------------

                 Key: MAPREDUCE-2370
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2370
             Project: Hadoop Map/Reduce
          Issue Type: Bug
          Components: job submission
    Affects Versions: 0.20.3
            Reporter: Tony Valderrama
            Priority: Minor


Due to the usage of URLDecoder in JobConf#findContainingJar, the path will be incorrectly modified if it contains the '+' character.  URLDecoder is intended for HTML form data (application/x-www-form-urlencoded), so all '+' will be converted to ' '.

This is easy to reproduce - install hadoop at a path which contains a '+' character and try to run a distcp job.  Mapreduce will fail to locate hadoop-tools.jar.  I have only investigated this error on 0.20.3-rc2.

Below is the exception which indicate the failure (correct path is /home/user/build/hadoop-core-0.20.3+8/hadoop-tools-0.20.3+8.jar):

Copy failed: java.io.FileNotFoundException: File /home/user/build/hadoop-core-0.20.3 8/hadoop-tools-0.20.3 8.jar does not exist.
        at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:361)
        at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:245)
        at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:192)
        at org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:1189)
        at org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:1165)
        at org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:1137)
        at org.apache.hadoop.mapred.JobClient.configureCommandLineOptions(JobClient.java:657)
        at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:761)
        at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:730)
        at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1249)
        at org.apache.hadoop.tools.DistCp.copy(DistCp.java:651)
        at org.apache.hadoop.tools.DistCp.run(DistCp.java:857)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
        at org.apache.hadoop.tools.DistCp.main(DistCp.java:884)


--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira