hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From omal...@apache.org
Subject svn commit: r684919 - in /hadoop/core/branches/branch-0.17: CHANGES.txt src/java/org/apache/hadoop/mapred/MapTask.java
Date Mon, 11 Aug 2008 20:52:37 GMT
Author: omalley
Date: Mon Aug 11 13:52:37 2008
New Revision: 684919

URL: http://svn.apache.org/viewvc?rev=684919&view=rev
Log:
HADOOP-3931. Fix corner case in the map-side sort that causes some values 
to be counted as too large and cause pre-mature spills to disk. Some values
will also bypass the combiner incorrectly. (cdouglas via omalley)

Modified:
    hadoop/core/branches/branch-0.17/CHANGES.txt
    hadoop/core/branches/branch-0.17/src/java/org/apache/hadoop/mapred/MapTask.java

Modified: hadoop/core/branches/branch-0.17/CHANGES.txt
URL: http://svn.apache.org/viewvc/hadoop/core/branches/branch-0.17/CHANGES.txt?rev=684919&r1=684918&r2=684919&view=diff
==============================================================================
--- hadoop/core/branches/branch-0.17/CHANGES.txt (original)
+++ hadoop/core/branches/branch-0.17/CHANGES.txt Mon Aug 11 13:52:37 2008
@@ -38,6 +38,10 @@
     HADOOP-3859. Allow the maximum number of xceivers in the data node to
     be configurable. (Johan Oskarsson via omalley)
 
+    HADOOP-3931. Fix corner case in the map-side sort that causes some values 
+    to be counted as too large and cause pre-mature spills to disk. Some values
+    will also bypass the combiner incorrectly. (cdouglas via omalley)
+
 Release 0.17.1 - 2008-06-23
 
   INCOMPATIBLE CHANGES

Modified: hadoop/core/branches/branch-0.17/src/java/org/apache/hadoop/mapred/MapTask.java
URL: http://svn.apache.org/viewvc/hadoop/core/branches/branch-0.17/src/java/org/apache/hadoop/mapred/MapTask.java?rev=684919&r1=684918&r2=684919&view=diff
==============================================================================
--- hadoop/core/branches/branch-0.17/src/java/org/apache/hadoop/mapred/MapTask.java (original)
+++ hadoop/core/branches/branch-0.17/src/java/org/apache/hadoop/mapred/MapTask.java Mon Aug
11 13:52:37 2008
@@ -435,7 +435,7 @@
       try {
         int keystart = bufindex;
         keySerializer.serialize(key);
-        if (bufindex < keystart || bufindex == bufvoid) {
+        if (bufindex < keystart) {
           // wrapped the key; reset required
           bb.reset();
           keystart = 0;



Mime
View raw message