hadoop-common-commits mailing list archives

Author: acmurthy
Date: Thu Jul  5 07:12:02 2012
New Revision: 1357478

URL: http://svn.apache.org/viewvc?rev=1357478&view=rev
Log:
MAPREDUCE-4368. Fix TaskRunner to deal with java.library.path with a quoted path on Windows.
Contributed by John Gordon. 

Modified:
    hadoop/common/branches/branch-1-win/CHANGES.txt
    hadoop/common/branches/branch-1-win/src/mapred/org/apache/hadoop/mapred/TaskRunner.java

Modified: hadoop/common/branches/branch-1-win/CHANGES.txt
URL: http://svn.apache.org/viewvc/hadoop/common/branches/branch-1-win/CHANGES.txt?rev=1357478&r1=1357477&r2=1357478&view=diff
==============================================================================
--- hadoop/common/branches/branch-1-win/CHANGES.txt (original)
+++ hadoop/common/branches/branch-1-win/CHANGES.txt Thu Jul  5 07:12:02 2012
@@ -56,6 +56,9 @@ branch-hadoop-1-win - unreleased
     MAPREDUCE-4369. Fix streaming test failures caused by
     WindowsResourceCalculatorPlugin. (Bikas Saha via acmurthy) 
 
+    MAPREDUCE-4368. Fix TaskRunner to deal with java.library.path with a
+    quoted path on Windows. (John Gordon via acmurthy) 
+
 Release 1.1.0 - unreleased
 
   NEW FEATURES

Modified: hadoop/common/branches/branch-1-win/src/mapred/org/apache/hadoop/mapred/TaskRunner.java
URL: http://svn.apache.org/viewvc/hadoop/common/branches/branch-1-win/src/mapred/org/apache/hadoop/mapred/TaskRunner.java?rev=1357478&r1=1357477&r2=1357478&view=diff
==============================================================================
--- hadoop/common/branches/branch-1-win/src/mapred/org/apache/hadoop/mapred/TaskRunner.java
(original)
+++ hadoop/common/branches/branch-1-win/src/mapred/org/apache/hadoop/mapred/TaskRunner.java
Thu Jul  5 07:12:02 2012
@@ -429,20 +429,30 @@ abstract class TaskRunner extends Thread
     } else {
       libraryPath += SYSTEM_PATH_SEPARATOR + workDir;
     }
+
+    // MAPREDUCE-4368
+    // For Windows, " is not a valid character for filenames, but
+    // is often embedded in concatenated path strings for paths
+    // with embedded spaces.  We will wrap the entire path in ",
+    // and embedded " marks cause matching issues and failure to
+    // launch the jar.
+    if (Shell.WINDOWS) {
+      libraryPath = libraryPath.replaceAll("\"", "");
+    }
+
     boolean hasUserLDPath = false;
     for(int i=0; i<javaOptsSplit.length ;i++) { 
       if(javaOptsSplit[i].startsWith("-Djava.library.path=")) {
         javaOptsSplit[i] += SYSTEM_PATH_SEPARATOR + libraryPath;
+        // MAPREDUCE-4377 blocks complete fix for 4368, since this is only
+        // partial library path in the case of quote-escaped strings
         hasUserLDPath = true;
-        break;
       }
+      vargs.add(javaOptsSplit[i]);
     }
     if(!hasUserLDPath) {
       vargs.add("-Djava.library.path=" + libraryPath);
     }
-    for (int i = 0; i < javaOptsSplit.length; i++) {
-      vargs.add(javaOptsSplit[i]);
-    }
 
     Path childTmpDir = createChildTmpDir(workDir, conf, false);
     vargs.add("-Djava.io.tmpdir=" + childTmpDir);