hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jl...@apache.org
Subject hadoop git commit: HADOOP-14902. LoadGenerator#genFile write close timing is incorrectly calculated. Contributed by Hanisha Koneru
Date Thu, 28 Sep 2017 21:48:28 GMT
Repository: hadoop
Updated Branches:
  refs/heads/branch-2.7 150222c0c -> ffe42e62c


HADOOP-14902. LoadGenerator#genFile write close timing is incorrectly calculated. Contributed
by Hanisha Koneru

(cherry picked from commit 6f789fe05766a61b12ca10df3f26ee354eac84aa)

Conflicts:
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/loadGenerator/LoadGenerator.java


Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo
Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/ffe42e62
Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/ffe42e62
Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/ffe42e62

Branch: refs/heads/branch-2.7
Commit: ffe42e62c994670ce2bfa83ee961d1651d14b026
Parents: 150222c
Author: Jason Lowe <jlowe@apache.org>
Authored: Thu Sep 28 16:38:30 2017 -0500
Committer: Jason Lowe <jlowe@apache.org>
Committed: Thu Sep 28 16:48:03 2017 -0500

----------------------------------------------------------------------
 hadoop-common-project/hadoop-common/CHANGES.txt          |  6 ++++++
 .../apache/hadoop/fs/loadGenerator/LoadGenerator.java    | 11 ++++++++---
 2 files changed, 14 insertions(+), 3 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/hadoop/blob/ffe42e62/hadoop-common-project/hadoop-common/CHANGES.txt
----------------------------------------------------------------------
diff --git a/hadoop-common-project/hadoop-common/CHANGES.txt b/hadoop-common-project/hadoop-common/CHANGES.txt
index 928ae54..f205288 100644
--- a/hadoop-common-project/hadoop-common/CHANGES.txt
+++ b/hadoop-common-project/hadoop-common/CHANGES.txt
@@ -20,6 +20,12 @@ Release 2.7.5 - UNRELEASED
     HADOOP-14702. Fix formatting issue and regression caused by conversion from
     APT to Markdown. (Doris Gu via iwasakims)
 
+    HADOOP-14881. LoadGenerator should use Time.monotonicNow() to measure
+    durations. (Bharat Viswanadham via jlowe)
+
+    HADOOP-14902. LoadGenerator#genFile write close timing is incorrectly
+    calculated. (Hanisha Koneru via jlowe)
+
 Release 2.7.4 - 2017-08-04
 
   INCOMPATIBLE CHANGES

http://git-wip-us.apache.org/repos/asf/hadoop/blob/ffe42e62/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/loadGenerator/LoadGenerator.java
----------------------------------------------------------------------
diff --git a/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/loadGenerator/LoadGenerator.java
b/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/loadGenerator/LoadGenerator.java
index 740100a..8f00d91 100644
--- a/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/loadGenerator/LoadGenerator.java
+++ b/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/loadGenerator/LoadGenerator.java
@@ -322,6 +322,7 @@ public class LoadGenerator extends Configured implements Tool {
     private void genFile(Path file, long fileSize) throws IOException {
       long startTimestamp = Time.monotonicNow();
       FSDataOutputStream out = null;
+      boolean isOutClosed = false;
       try {
         out = fc.create(file,
             EnumSet.of(CreateFlag.CREATE, CreateFlag.OVERWRITE),
@@ -337,11 +338,15 @@ public class LoadGenerator extends Configured implements Tool {
           i -= s;
         }
 
-        startTimestamp = Time.monotonicNow();
-        executionTime[WRITE_CLOSE] += (Time.monotonicNow() - startTimestamp);
+        startTime = Time.monotonicNow();
+        out.close();
+        executionTime[WRITE_CLOSE] += (Time.monotonicNow() - startTime);
         numOfOps[WRITE_CLOSE]++;
+        isOutClosed = true;
       } finally {
-        IOUtils.cleanup(LOG, out);
+        if (!isOutClosed && out != null) {
+          out.close();
+        }
       }
     }
   }


---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-commits-help@hadoop.apache.org


Mime
View raw message