hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From tomwh...@apache.org
Subject svn commit: r531084 - in /lucene/hadoop/trunk: CHANGES.txt src/java/org/apache/hadoop/mapred/ReduceTask.java
Date Sat, 21 Apr 2007 18:55:39 GMT
Author: tomwhite
Date: Sat Apr 21 11:55:38 2007
New Revision: 531084

URL: http://svn.apache.org/viewvc?view=rev&rev=531084
Log:
HADOOP-1152.  Fix race condition in MapOutputCopier.copyOutput file rename causing possible
reduce task hang.  Contributed by Tahir Hashmi.

Modified:
    lucene/hadoop/trunk/CHANGES.txt
    lucene/hadoop/trunk/src/java/org/apache/hadoop/mapred/ReduceTask.java

Modified: lucene/hadoop/trunk/CHANGES.txt
URL: http://svn.apache.org/viewvc/lucene/hadoop/trunk/CHANGES.txt?view=diff&rev=531084&r1=531083&r2=531084
==============================================================================
--- lucene/hadoop/trunk/CHANGES.txt (original)
+++ lucene/hadoop/trunk/CHANGES.txt Sat Apr 21 11:55:38 2007
@@ -234,6 +234,10 @@
 70. HADOOP-1275.  Fix misspelled job notification property in
     hadoop-default.xml.  (Alejandro Abdelnur via tomwhite)
 
+71. HADOOP-1152.  Fix race condition in MapOutputCopier.copyOutput file
+    rename causing possible reduce task hang.
+    (Tahir Hashmi via tomwhite)
+
 
 Release 0.12.3 - 2007-04-06
 

Modified: lucene/hadoop/trunk/src/java/org/apache/hadoop/mapred/ReduceTask.java
URL: http://svn.apache.org/viewvc/lucene/hadoop/trunk/src/java/org/apache/hadoop/mapred/ReduceTask.java?view=diff&rev=531084&r1=531083&r2=531084
==============================================================================
--- lucene/hadoop/trunk/src/java/org/apache/hadoop/mapred/ReduceTask.java (original)
+++ lucene/hadoop/trunk/src/java/org/apache/hadoop/mapred/ReduceTask.java Sat Apr 21 11:55:38
2007
@@ -698,14 +698,17 @@
             fs.delete(tmpFilename);
             return CopyResult.OBSOLETE;
           }
+          
+          bytes = fs.getLength(tmpFilename);
           // if we can't rename the file, something is broken (and IOException
           // will be thrown). 
           if (!fs.rename(tmpFilename, finalFilename)) {
             fs.delete(tmpFilename);
+            bytes = -1;
             throw new IOException("failure to rename map output " + 
                                   tmpFilename);
           }
-          bytes = fs.getLength(finalFilename);
+          
           LOG.info(reduceId + " done copying " + loc.getMapTaskId() +
                    " output from " + loc.getHost() + ".");
           //Create a thread to do merges. Synchronize access/update to 



Mime
View raw message