Return-Path: X-Original-To: apmail-falcon-commits-archive@minotaur.apache.org Delivered-To: apmail-falcon-commits-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 9C43D1163E for ; Mon, 7 Jul 2014 11:54:50 +0000 (UTC) Received: (qmail 98228 invoked by uid 500); 7 Jul 2014 11:54:50 -0000 Delivered-To: apmail-falcon-commits-archive@falcon.apache.org Received: (qmail 98191 invoked by uid 500); 7 Jul 2014 11:54:50 -0000 Mailing-List: contact commits-help@falcon.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@falcon.incubator.apache.org Delivered-To: mailing list commits@falcon.incubator.apache.org Received: (qmail 98182 invoked by uid 99); 7 Jul 2014 11:54:50 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 07 Jul 2014 11:54:50 +0000 X-ASF-Spam-Status: No, hits=-2000.7 required=5.0 tests=ALL_TRUSTED,RP_MATCHES_RCVD X-Spam-Check-By: apache.org Received: from [140.211.11.3] (HELO mail.apache.org) (140.211.11.3) by apache.org (qpsmtpd/0.29) with SMTP; Mon, 07 Jul 2014 11:54:48 +0000 Received: (qmail 97176 invoked by uid 99); 7 Jul 2014 11:54:27 -0000 Received: from tyr.zones.apache.org (HELO tyr.zones.apache.org) (140.211.11.114) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 07 Jul 2014 11:54:27 +0000 Received: by tyr.zones.apache.org (Postfix, from userid 65534) id 3101D9A74A7; Mon, 7 Jul 2014 11:54:27 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: shwethags@apache.org To: commits@falcon.incubator.apache.org Message-Id: <048cbf9e34524727a42d7cf42e34abf2@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: git commit: FALCON-496 Feed Replication with Empty Directories giving error. Contributed by pavan kumar kolamuri Date: Mon, 7 Jul 2014 11:54:27 +0000 (UTC) X-Virus-Checked: Checked by ClamAV on apache.org Repository: incubator-falcon Updated Branches: refs/heads/master 5517156e1 -> cfd647af0 FALCON-496 Feed Replication with Empty Directories giving error. Contributed by pavan kumar kolamuri Project: http://git-wip-us.apache.org/repos/asf/incubator-falcon/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-falcon/commit/cfd647af Tree: http://git-wip-us.apache.org/repos/asf/incubator-falcon/tree/cfd647af Diff: http://git-wip-us.apache.org/repos/asf/incubator-falcon/diff/cfd647af Branch: refs/heads/master Commit: cfd647af0e526d1de45b95fe7506157f868306d5 Parents: 5517156 Author: Shwetha GS Authored: Mon Jul 7 17:24:16 2014 +0530 Committer: Shwetha GS Committed: Mon Jul 7 17:24:16 2014 +0530 ---------------------------------------------------------------------- CHANGES.txt | 3 +++ .../org/apache/falcon/replication/FeedReplicator.java | 14 ++++++++++++-- 2 files changed, 15 insertions(+), 2 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-falcon/blob/cfd647af/CHANGES.txt ---------------------------------------------------------------------- diff --git a/CHANGES.txt b/CHANGES.txt index b27964b..b8e22bd 100755 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -20,6 +20,9 @@ Trunk (Unreleased) OPTIMIZATIONS BUG FIXES + FALCON-496 Feed Replication with Empty Directories giving error. + (pavan kumar kolamuri via Shwetha GS) + FALCON-483 Fix the failing test ConfigurationStoreTest.testConcurrentRemoves on jenkins. (Sowmya Ramesh via Shwetha GS) http://git-wip-us.apache.org/repos/asf/incubator-falcon/blob/cfd647af/replication/src/main/java/org/apache/falcon/replication/FeedReplicator.java ---------------------------------------------------------------------- diff --git a/replication/src/main/java/org/apache/falcon/replication/FeedReplicator.java b/replication/src/main/java/org/apache/falcon/replication/FeedReplicator.java index 43cfed9..f01c46c 100644 --- a/replication/src/main/java/org/apache/falcon/replication/FeedReplicator.java +++ b/replication/src/main/java/org/apache/falcon/replication/FeedReplicator.java @@ -18,6 +18,7 @@ package org.apache.falcon.replication; import org.apache.commons.cli.*; +import org.apache.commons.lang.StringUtils; import org.apache.falcon.entity.EntityUtil; import org.apache.falcon.entity.Storage; import org.apache.hadoop.conf.Configuration; @@ -142,14 +143,23 @@ public class FeedReplicator extends Configured implements Tool { String relativePath = includePath.toString().substring(sourcePath.toString().length()); String fixedPath = getFixedPath(relativePath); - FileStatus[] files = fs.globStatus(new Path(targetPath.toString() + "/" + fixedPath)); + Path finalOutputPath; + if (StringUtils.isNotEmpty(fixedPath)) { + finalOutputPath = new Path(targetPath, fixedPath); + } else { + finalOutputPath = targetPath; + } + + FileStatus[] files = fs.globStatus(finalOutputPath); if (files != null) { for (FileStatus file : files) { fs.create(new Path(file.getPath(), EntityUtil.SUCCEEDED_FILE_NAME)).close(); LOG.info("Created {}", new Path(file.getPath(), EntityUtil.SUCCEEDED_FILE_NAME)); } } else { - LOG.info("No files present in path: {}", new Path(targetPath, fixedPath)); + // As distcp is not copying empty directories we are creating _SUCCESS file here + fs.create(new Path(finalOutputPath, EntityUtil.SUCCEEDED_FILE_NAME)).close(); + LOG.info("No files present in path: {}", finalOutputPath); } }