Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id E49FC2009A8 for ; Tue, 17 May 2016 14:04:13 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id E35F9160A20; Tue, 17 May 2016 12:04:13 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 3A43A1609AE for ; Tue, 17 May 2016 14:04:13 +0200 (CEST) Received: (qmail 28871 invoked by uid 500); 17 May 2016 12:04:12 -0000 Mailing-List: contact common-commits-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list common-commits@hadoop.apache.org Received: (qmail 28862 invoked by uid 99); 17 May 2016 12:04:12 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 17 May 2016 12:04:12 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 3725EDFB93; Tue, 17 May 2016 12:04:12 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: stevel@apache.org To: common-commits@hadoop.apache.org Message-Id: X-Mailer: ASF-Git Admin Mailer Subject: hadoop git commit: HADOOP-13163 Reuse pre-computed filestatus in Distcp-CopyMapper (Rajesh Balamohan via stevel) Date: Tue, 17 May 2016 12:04:12 +0000 (UTC) archived-at: Tue, 17 May 2016 12:04:14 -0000 Repository: hadoop Updated Branches: refs/heads/branch-2 19db965c2 -> dfc062a09 HADOOP-13163 Reuse pre-computed filestatus in Distcp-CopyMapper (Rajesh Balamohan via stevel) Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/dfc062a0 Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/dfc062a0 Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/dfc062a0 Branch: refs/heads/branch-2 Commit: dfc062a0927a5f5c6db9957cd738a978d97b9705 Parents: 19db965 Author: Steve Loughran Authored: Tue May 17 12:59:59 2016 +0100 Committer: Steve Loughran Committed: Tue May 17 13:01:50 2016 +0100 ---------------------------------------------------------------------- .../java/org/apache/hadoop/tools/mapred/CopyMapper.java | 10 ++-------- 1 file changed, 2 insertions(+), 8 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/hadoop/blob/dfc062a0/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java ---------------------------------------------------------------------- diff --git a/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java b/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java index 6f88660..a979a29 100644 --- a/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java +++ b/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java @@ -245,7 +245,7 @@ public class CopyMapper extends Mapper return; } - FileAction action = checkUpdate(sourceFS, sourceCurrStatus, target); + FileAction action = checkUpdate(sourceFS, sourceCurrStatus, target, targetStatus); if (action == FileAction.SKIP) { LOG.info("Skipping copy of " + sourceCurrStatus.getPath() + " to " + target); @@ -332,13 +332,7 @@ public class CopyMapper extends Mapper } private FileAction checkUpdate(FileSystem sourceFS, FileStatus source, - Path target) throws IOException { - final FileStatus targetFileStatus; - try { - targetFileStatus = targetFS.getFileStatus(target); - } catch (FileNotFoundException e) { - return FileAction.OVERWRITE; - } + Path target, FileStatus targetFileStatus) throws IOException { if (targetFileStatus != null && !overWrite) { if (canSkip(sourceFS, source, targetFileStatus)) { return FileAction.SKIP; --------------------------------------------------------------------- To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org For additional commands, e-mail: common-commits-help@hadoop.apache.org