hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ste...@apache.org
Subject hadoop git commit: HADOOP-13163 Reuse pre-computed filestatus in Distcp-CopyMapper (Rajesh Balamohan via stevel)
Date Tue, 17 May 2016 12:03:16 GMT
Repository: hadoop
Updated Branches:
  refs/heads/branch-2.8 fc221ecf3 -> b44ecfff1


HADOOP-13163 Reuse pre-computed filestatus in Distcp-CopyMapper (Rajesh Balamohan via stevel)


Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo
Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/b44ecfff
Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/b44ecfff
Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/b44ecfff

Branch: refs/heads/branch-2.8
Commit: b44ecfff121ae36f7436862223d5b6379e2a58ff
Parents: fc221ec
Author: Steve Loughran <stevel@apache.org>
Authored: Tue May 17 12:59:59 2016 +0100
Committer: Steve Loughran <stevel@apache.org>
Committed: Tue May 17 13:02:22 2016 +0100

----------------------------------------------------------------------
 .../java/org/apache/hadoop/tools/mapred/CopyMapper.java   | 10 ++--------
 1 file changed, 2 insertions(+), 8 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/hadoop/blob/b44ecfff/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java
----------------------------------------------------------------------
diff --git a/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java
b/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java
index cca36df..c456cc5 100644
--- a/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java
+++ b/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java
@@ -243,7 +243,7 @@ public class CopyMapper extends Mapper<Text, CopyListingFileStatus,
Text, Text>
         return;
       }
 
-      FileAction action = checkUpdate(sourceFS, sourceCurrStatus, target);
+      FileAction action = checkUpdate(sourceFS, sourceCurrStatus, target, targetStatus);
       if (action == FileAction.SKIP) {
         LOG.info("Skipping copy of " + sourceCurrStatus.getPath()
                  + " to " + target);
@@ -330,13 +330,7 @@ public class CopyMapper extends Mapper<Text, CopyListingFileStatus,
Text, Text>
   }
 
   private FileAction checkUpdate(FileSystem sourceFS, FileStatus source,
-      Path target) throws IOException {
-    final FileStatus targetFileStatus;
-    try {
-      targetFileStatus = targetFS.getFileStatus(target);
-    } catch (FileNotFoundException e) {
-      return FileAction.OVERWRITE;
-    }
+      Path target, FileStatus targetFileStatus) throws IOException {
     if (targetFileStatus != null && !overWrite) {
       if (canSkip(sourceFS, source, targetFileStatus)) {
         return FileAction.SKIP;


---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-commits-help@hadoop.apache.org


Mime
View raw message