hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From cutt...@apache.org
Subject svn commit: r416062 - in /lucene/hadoop/trunk: CHANGES.txt src/java/org/apache/hadoop/dfs/DFSClient.java
Date Wed, 21 Jun 2006 18:35:32 GMT
Author: cutting
Date: Wed Jun 21 11:35:32 2006
New Revision: 416062

URL: http://svn.apache.org/viewvc?rev=416062&view=rev
Log:
HADOOP-311.  Change DFS client to retry failed reads, so that a single read failure alone
will not cause failure of a task.  Contributed by Owen.

Modified:
    lucene/hadoop/trunk/CHANGES.txt
    lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/DFSClient.java

Modified: lucene/hadoop/trunk/CHANGES.txt
URL: http://svn.apache.org/viewvc/lucene/hadoop/trunk/CHANGES.txt?rev=416062&r1=416061&r2=416062&view=diff
==============================================================================
--- lucene/hadoop/trunk/CHANGES.txt (original)
+++ lucene/hadoop/trunk/CHANGES.txt Wed Jun 21 11:35:32 2006
@@ -26,6 +26,10 @@
     protocol: clients and servers must both be upgraded to the new
     version to ensure correct operation.  (Devaraj Das via cutting)
 
+ 7. HADOOP-311.  Change DFS client to retry failed reads, so that a
+    single read failure will not alone cause failure of a task.
+    (omalley via cutting)
+
 
 Release 0.3.2 - 2006-06-09
 

Modified: lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/DFSClient.java
URL: http://svn.apache.org/viewvc/lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/DFSClient.java?rev=416062&r1=416061&r2=416062&view=diff
==============================================================================
--- lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/DFSClient.java (original)
+++ lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/DFSClient.java Wed Jun 21 11:35:32
2006
@@ -634,14 +634,26 @@
                 throw new IOException("Stream closed");
             }
             if (pos < filelen) {
-                if (pos > blockEnd) {
-                    blockSeekTo(pos);
+              int retries = 2;
+              while (retries > 0) {
+                try {
+                  if (pos > blockEnd) {
+                      blockSeekTo(pos);
+                  }
+                  int realLen = Math.min(len, (int) (blockEnd - pos + 1));
+                  int result = blockStream.read(buf, off, realLen);
+                  if (result >= 0) {
+                      pos += result;
+                  }
+                  return result;
+                } catch (IOException e) {
+                  LOG.warn("DFS Read: " + StringUtils.stringifyException(e));
+                  blockEnd = -1;
+                  if (--retries == 0) {
+                    throw e;
+                  }
                 }
-                int result = blockStream.read(buf, off, Math.min(len, (int) (blockEnd - pos
+ 1)));
-                if (result >= 0) {
-                    pos += result;
-                }
-                return result;
+              }
             }
             return -1;
         }



Mime
View raw message