hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From iwasak...@apache.org
Subject hadoop git commit: HADOOP-14405. Fix performance regression due to incorrect use of DataChecksum. Contributed by LiXin Ge.
Date Wed, 10 May 2017 05:17:20 GMT
Repository: hadoop
Updated Branches:
  refs/heads/trunk ab2bb93a2 -> 06ffeb8d4


HADOOP-14405. Fix performance regression due to incorrect use of DataChecksum. Contributed
by LiXin Ge.


Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo
Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/06ffeb8d
Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/06ffeb8d
Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/06ffeb8d

Branch: refs/heads/trunk
Commit: 06ffeb8d4d705d14e8755364099286ac4ed3bb83
Parents: ab2bb93
Author: Masatake Iwasaki <iwasakims@apache.org>
Authored: Wed May 10 14:14:20 2017 +0900
Committer: Masatake Iwasaki <iwasakims@apache.org>
Committed: Wed May 10 14:16:41 2017 +0900

----------------------------------------------------------------------
 .../java/org/apache/hadoop/util/DataChecksum.java     | 14 +++++++++++---
 1 file changed, 11 insertions(+), 3 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/hadoop/blob/06ffeb8d/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/DataChecksum.java
----------------------------------------------------------------------
diff --git a/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/DataChecksum.java
b/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/DataChecksum.java
index 6982a92..03946af 100644
--- a/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/DataChecksum.java
+++ b/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/DataChecksum.java
@@ -296,12 +296,20 @@ public class DataChecksum implements Checksum {
   public void verifyChunkedSums(ByteBuffer data, ByteBuffer checksums,
       String fileName, long basePos) throws ChecksumException {
     if (type.size == 0) return;
-    
+
     if (data.hasArray() && checksums.hasArray()) {
       final int dataOffset = data.arrayOffset() + data.position();
       final int crcsOffset = checksums.arrayOffset() + checksums.position();
-      verifyChunked(type, summer, data.array(), dataOffset, data.remaining(),
-          bytesPerChecksum, checksums.array(), crcsOffset, fileName, basePos);
+
+      if (NativeCrc32.isAvailable()) {
+        NativeCrc32.verifyChunkedSumsByteArray(bytesPerChecksum, type.id,
+                checksums.array(), crcsOffset, data.array(), dataOffset,
+                data.remaining(), fileName, basePos);
+      } else {
+        verifyChunked(type, summer, data.array(), dataOffset, data.remaining(),
+                bytesPerChecksum, checksums.array(), crcsOffset, fileName,
+                basePos);
+      }
       return;
     }
     if (NativeCrc32.isAvailable() && data.isDirect()) {


---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-commits-help@hadoop.apache.org


Mime
View raw message