Mailing-List: contact core-commits-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: core-dev@hadoop.apache.org
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Subject: svn commit: r722325 - in /hadoop/core/branches/branch-0.18:
 CHANGES.txt
 src/c++/librecordio/binarchive.cc
Date: Tue, 02 Dec 2008 01:37:48 -0000
To: core-commits@hadoop.apache.org
From: cdouglas@apache.org
Message-Id: <20081202013748.B69CA2388975@eris.apache.org>

Author: cdouglas
Date: Mon Dec  1 17:37:48 2008
New Revision: 722325

URL: http://svn.apache.org/viewvc?rev=722325&view=rev
Log:
HADOOP-4713. Fix librecordio to handle records larger than 64k. Contributed by Christian Kunz.

Modified:
    hadoop/core/branches/branch-0.18/CHANGES.txt
    hadoop/core/branches/branch-0.18/src/c++/librecordio/binarchive.cc

Modified: hadoop/core/branches/branch-0.18/CHANGES.txt
URL: http://svn.apache.org/viewvc/hadoop/core/branches/branch-0.18/CHANGES.txt?rev=722325&r1=722324&r2=722325&view=diff
==============================================================================
--- hadoop/core/branches/branch-0.18/CHANGES.txt (original)
+++ hadoop/core/branches/branch-0.18/CHANGES.txt Mon Dec  1 17:37:48 2008
@@ -52,6 +52,9 @@
     HADOOP-4257. The DFS client should pick only one datanode as the candidate
     to initiate lease recovery.  (Tsz Wo (Nicholas), SZE via dhruba)
 
+    HADOOP-4713. Fix librecordio to handle records larger than 64k. (Christian
+    Kunz via cdouglas)
+
 Release 0.18.2 - 2008-11-03
 
   BUG FIXES

Modified: hadoop/core/branches/branch-0.18/src/c++/librecordio/binarchive.cc
URL: http://svn.apache.org/viewvc/hadoop/core/branches/branch-0.18/src/c%2B%2B/librecordio/binarchive.cc?rev=722325&r1=722324&r2=722325&view=diff
==============================================================================
--- hadoop/core/branches/branch-0.18/src/c++/librecordio/binarchive.cc (original)
+++ hadoop/core/branches/branch-0.18/src/c++/librecordio/binarchive.cc Mon Dec  1 17:37:48 2008
@@ -161,10 +161,19 @@
   int32_t len = 0;
   ::deserializeInt(len, stream);
   if (len > 0) {
-    char buf[len];
-    stream.read((void*) buf, len);
-    std::string s(buf, len);
-    t = s;
+    // resize the string to the right length
+    t.resize(len);
+    // read into the string in 64k chunks
+    const int bufSize = 65536;
+    int offset = 0;
+    char buf[bufSize];
+    while (len > 0) {
+      int chunkLength = len > bufSize ? bufSize : len;
+      stream.read((void *)buf, chunkLength);
+      t.replace(offset, chunkLength, buf, chunkLength);
+      offset += chunkLength;
+      len -= chunkLength;
+    }
   }
 }