Return-Path: X-Original-To: apmail-falcon-commits-archive@minotaur.apache.org Delivered-To: apmail-falcon-commits-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 2F18F11552 for ; Fri, 5 Sep 2014 07:25:55 +0000 (UTC) Received: (qmail 72414 invoked by uid 500); 5 Sep 2014 07:25:55 -0000 Delivered-To: apmail-falcon-commits-archive@falcon.apache.org Received: (qmail 72379 invoked by uid 500); 5 Sep 2014 07:25:55 -0000 Mailing-List: contact commits-help@falcon.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@falcon.incubator.apache.org Delivered-To: mailing list commits@falcon.incubator.apache.org Received: (qmail 72369 invoked by uid 99); 5 Sep 2014 07:25:55 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 05 Sep 2014 07:25:55 +0000 X-ASF-Spam-Status: No, hits=-2001.7 required=5.0 tests=ALL_TRUSTED,RP_MATCHES_RCVD X-Spam-Check-By: apache.org Received: from [140.211.11.3] (HELO mail.apache.org) (140.211.11.3) by apache.org (qpsmtpd/0.29) with SMTP; Fri, 05 Sep 2014 07:25:50 +0000 Received: (qmail 70522 invoked by uid 99); 5 Sep 2014 07:25:30 -0000 Received: from tyr.zones.apache.org (HELO tyr.zones.apache.org) (140.211.11.114) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 05 Sep 2014 07:25:30 +0000 Received: by tyr.zones.apache.org (Postfix, from userid 65534) id 0D484A09A99; Fri, 5 Sep 2014 07:25:30 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: rostafiychuk@apache.org To: commits@falcon.incubator.apache.org Message-Id: X-Mailer: ASF-Git Admin Mailer Subject: git commit: FALCON-657 String datePattern moved to BaseTestClass in falcon-regression Date: Fri, 5 Sep 2014 07:25:30 +0000 (UTC) X-Virus-Checked: Checked by ClamAV on apache.org Repository: incubator-falcon Updated Branches: refs/heads/master cb11ff259 -> be08a2e70 FALCON-657 String datePattern moved to BaseTestClass in falcon-regression Project: http://git-wip-us.apache.org/repos/asf/incubator-falcon/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-falcon/commit/be08a2e7 Tree: http://git-wip-us.apache.org/repos/asf/incubator-falcon/tree/be08a2e7 Diff: http://git-wip-us.apache.org/repos/asf/incubator-falcon/diff/be08a2e7 Branch: refs/heads/master Commit: be08a2e709956c32cb13e0ec66aff729a7ee234b Parents: cb11ff2 Author: Ruslan Ostafiychuk Authored: Thu Sep 4 14:48:19 2014 +0300 Committer: Ruslan Ostafiychuk Committed: Fri Sep 5 10:23:35 2014 +0300 ---------------------------------------------------------------------- falcon-regression/CHANGES.txt | 2 + .../regression/testHelper/BaseTestClass.java | 1 + .../falcon/regression/AuthorizationTest.java | 15 ++++--- .../regression/ELExp_FutureAndLatestTest.java | 7 ++-- .../regression/EmbeddedPigScriptTest.java | 5 +-- .../regression/FeedInstanceStatusTest.java | 2 +- .../falcon/regression/FeedReplicationTest.java | 6 +-- .../falcon/regression/InstanceParamTest.java | 3 +- .../falcon/regression/InstanceSummaryTest.java | 3 +- .../apache/falcon/regression/NewRetryTest.java | 4 +- .../falcon/regression/NoOutputProcessTest.java | 2 +- .../ProcessInstanceColoMixedTest.java | 3 +- .../regression/ProcessInstanceKillsTest.java | 5 +-- .../regression/ProcessInstanceRerunTest.java | 7 ++-- .../regression/ProcessInstanceResumeTest.java | 6 +-- .../regression/ProcessInstanceRunningTest.java | 6 +-- .../regression/ProcessInstanceStatusTest.java | 12 +++--- .../regression/ProcessInstanceSuspendTest.java | 6 +-- .../falcon/regression/ProcessLibPathTest.java | 7 ++-- .../lineage/LineageApiProcessInstanceTest.java | 5 +-- .../regression/lineage/LineageApiTest.java | 5 +-- .../regression/prism/EntityDryRunTest.java | 6 +-- .../prism/FeedDelayParallelTimeoutTest.java | 2 +- .../prism/NewPrismProcessUpdateTest.java | 2 +- .../regression/prism/PrismFeedDeleteTest.java | 18 +++----- .../prism/PrismFeedLateReplicationTest.java | 3 +- .../PrismFeedReplicationPartitionExpTest.java | 43 ++++++++++---------- .../prism/PrismFeedReplicationUpdateTest.java | 13 +++--- .../regression/prism/PrismFeedSnSTest.java | 4 +- .../regression/prism/PrismFeedUpdateTest.java | 6 +-- .../prism/RescheduleKilledProcessTest.java | 3 +- .../RescheduleProcessInFinalStatesTest.java | 5 +-- .../prism/UpdateAtSpecificTimeTest.java | 9 ++-- .../falcon/regression/ui/LineageGraphTest.java | 3 +- .../falcon/regression/ui/ProcessUITest.java | 7 ++-- 35 files changed, 99 insertions(+), 137 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-falcon/blob/be08a2e7/falcon-regression/CHANGES.txt ---------------------------------------------------------------------- diff --git a/falcon-regression/CHANGES.txt b/falcon-regression/CHANGES.txt index 20d5882..aecd520 100644 --- a/falcon-regression/CHANGES.txt +++ b/falcon-regression/CHANGES.txt @@ -9,6 +9,8 @@ Trunk (Unreleased) via Samarth Gupta) IMPROVEMENTS + FALCON-657 String datePattern moved to BaseTestClass (Ruslan Ostafiychuk) + FALCON-643 Tests with zero-output/input scenario amended to match test case (Paul Isaychuk via Ruslan Ostafiychuk) http://git-wip-us.apache.org/repos/asf/incubator-falcon/blob/be08a2e7/falcon-regression/merlin/src/main/java/org/apache/falcon/regression/testHelper/BaseTestClass.java ---------------------------------------------------------------------- diff --git a/falcon-regression/merlin/src/main/java/org/apache/falcon/regression/testHelper/BaseTestClass.java b/falcon-regression/merlin/src/main/java/org/apache/falcon/regression/testHelper/BaseTestClass.java index a951da7..c16c8f4 100644 --- a/falcon-regression/merlin/src/main/java/org/apache/falcon/regression/testHelper/BaseTestClass.java +++ b/falcon-regression/merlin/src/main/java/org/apache/falcon/regression/testHelper/BaseTestClass.java @@ -52,6 +52,7 @@ public class BaseTestClass { public String baseHDFSDir = "/tmp/falcon-regression"; public static final String PRISM_PREFIX = "prism"; protected Bundle[] bundles; + public static final String MINUTE_DATE_PATTERN = "/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}"; public BaseTestClass() { http://git-wip-us.apache.org/repos/asf/incubator-falcon/blob/be08a2e7/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/AuthorizationTest.java ---------------------------------------------------------------------- diff --git a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/AuthorizationTest.java b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/AuthorizationTest.java index fac215f..8bf4288 100644 --- a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/AuthorizationTest.java +++ b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/AuthorizationTest.java @@ -70,8 +70,7 @@ public class AuthorizationTest extends BaseTestClass { private OozieClient clusterOC = serverOC.get(0); private String baseTestDir = baseHDFSDir + "/AuthorizationTest"; private String aggregateWorkflowDir = baseTestDir + "/aggregator"; - private String datePattern = "/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}"; - private String feedInputPath = baseTestDir + "/input" + datePattern; + private String feedInputPath = baseTestDir + "/input" + MINUTE_DATE_PATTERN; @BeforeClass(alwaysRun = true) public void uploadWorkflow() throws Exception { @@ -538,8 +537,8 @@ public class AuthorizationTest extends BaseTestClass { .readEntityName(feed)) && !definition.contains("(feed) not found"), "Feed should be already submitted"); //update feed definition - String newFeed = Util.setFeedPathValue(feed, - baseHDFSDir + "/randomPath/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}/"); + String newFeed = Util.setFeedPathValue(feed, baseHDFSDir + "/randomPath" + + MINUTE_DATE_PATTERN); //try to update feed by U2 KerberosHelper.loginFromKeytab(MerlinConstants.USER2_NAME); final ServiceResponse serviceResponse = prism.getFeedHelper().update(feed, newFeed, @@ -560,8 +559,8 @@ public class AuthorizationTest extends BaseTestClass { Util.URLS.SUBMIT_AND_SCHEDULE_URL, feed)); AssertUtil.checkStatus(clusterOC, EntityType.FEED, feed, Job.Status.RUNNING); //update feed definition - String newFeed = Util.setFeedPathValue(feed, - baseHDFSDir + "/randomPath/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}/"); + String newFeed = Util.setFeedPathValue(feed, baseHDFSDir + "/randomPath" + + MINUTE_DATE_PATTERN); //try to update feed by U2 KerberosHelper.loginFromKeytab(MerlinConstants.USER2_NAME); final ServiceResponse serviceResponse = prism.getFeedHelper().update(feed, newFeed, @@ -651,8 +650,8 @@ public class AuthorizationTest extends BaseTestClass { .getLatestBundleID(cluster, Util.readEntityName(feed), EntityType.FEED); //update feed definition - String newFeed = Util.setFeedPathValue(feed, - baseHDFSDir + "/randomPath/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}/"); + String newFeed = Util.setFeedPathValue(feed, baseHDFSDir + "/randomPath" + + MINUTE_DATE_PATTERN); //update feed by U1 KerberosHelper.loginFromKeytab(MerlinConstants.CURRENT_USER_NAME); http://git-wip-us.apache.org/repos/asf/incubator-falcon/blob/be08a2e7/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/ELExp_FutureAndLatestTest.java ---------------------------------------------------------------------- diff --git a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/ELExp_FutureAndLatestTest.java b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/ELExp_FutureAndLatestTest.java index 63484c1..25e2dfe 100644 --- a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/ELExp_FutureAndLatestTest.java +++ b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/ELExp_FutureAndLatestTest.java @@ -67,8 +67,7 @@ public class ELExp_FutureAndLatestTest extends BaseTestClass { String startDate = TimeUtil.getTimeWrtSystemTime(-20); String endDate = TimeUtil.getTimeWrtSystemTime(70); - b.setInputFeedDataPath( - baseTestDir + "/ELExp_latest/testData/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}"); + b.setInputFeedDataPath(baseTestDir + "/ELExp_latest/testData" + MINUTE_DATE_PATTERN); b.setProcessWorkflow(aggregateWorkflowDir); List dataDates = TimeUtil.getMinuteDatesOnEitherSide(startDate, endDate, 1); @@ -83,8 +82,8 @@ public class ELExp_FutureAndLatestTest extends BaseTestClass { bundles[0] = BundleUtil.readELBundle(); bundles[0] = new Bundle(bundles[0], cluster); bundles[0].generateUniqueBundle(); - bundles[0].setInputFeedDataPath( - baseTestDir + "/ELExp_latest/testData/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}"); + bundles[0].setInputFeedDataPath(baseTestDir + "/ELExp_latest/testData" + + MINUTE_DATE_PATTERN); bundles[0].setInputFeedPeriodicity(5, TimeUnit.minutes); bundles[0].setInputFeedValidity("2010-04-01T00:00Z", "2015-04-01T00:00Z"); String processStart = TimeUtil.getTimeWrtSystemTime(-3); http://git-wip-us.apache.org/repos/asf/incubator-falcon/blob/be08a2e7/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/EmbeddedPigScriptTest.java ---------------------------------------------------------------------- diff --git a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/EmbeddedPigScriptTest.java b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/EmbeddedPigScriptTest.java index 00f68e6..a0986fa 100644 --- a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/EmbeddedPigScriptTest.java +++ b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/EmbeddedPigScriptTest.java @@ -66,7 +66,7 @@ public class EmbeddedPigScriptTest extends BaseTestClass { String pigTestDir = baseHDFSDir + "/EmbeddedPigScriptTest"; String pigScriptDir = pigTestDir + "/EmbeddedPigScriptTest/pig"; String pigScriptLocation = pigScriptDir + "/id.pig"; - String inputPath = pigTestDir + "/input/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}"; + String inputPath = pigTestDir + "/input" + MINUTE_DATE_PATTERN; private static final Logger logger = Logger.getLogger(EmbeddedPigScriptTest.class); private static final double TIMEOUT = 15; String processName; @@ -96,8 +96,7 @@ public class EmbeddedPigScriptTest extends BaseTestClass { bundles[0] = new Bundle(bundles[0], cluster); bundles[0].generateUniqueBundle(); bundles[0].setInputFeedDataPath(inputPath); - bundles[0].setOutputFeedLocationData( - pigTestDir + "/output-data/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}"); + bundles[0].setOutputFeedLocationData(pigTestDir + "/output-data" + MINUTE_DATE_PATTERN); bundles[0].setProcessWorkflow(pigScriptLocation); bundles[0].setProcessInputNames("INPUT"); bundles[0].setProcessOutputNames("OUTPUT"); http://git-wip-us.apache.org/repos/asf/incubator-falcon/blob/be08a2e7/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/FeedInstanceStatusTest.java ---------------------------------------------------------------------- diff --git a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/FeedInstanceStatusTest.java b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/FeedInstanceStatusTest.java index d4fab30..1e01101 100644 --- a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/FeedInstanceStatusTest.java +++ b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/FeedInstanceStatusTest.java @@ -52,7 +52,7 @@ import java.lang.reflect.Method; public class FeedInstanceStatusTest extends BaseTestClass { private String baseTestDir = baseHDFSDir + "/FeedInstanceStatusTest"; - private String feedInputPath = baseTestDir + "/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}/"; + private String feedInputPath = baseTestDir + MINUTE_DATE_PATTERN; private String aggregateWorkflowDir = baseTestDir + "/aggregator"; ColoHelper cluster2 = servers.get(1); http://git-wip-us.apache.org/repos/asf/incubator-falcon/blob/be08a2e7/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/FeedReplicationTest.java ---------------------------------------------------------------------- diff --git a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/FeedReplicationTest.java b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/FeedReplicationTest.java index c4c11cf..fa28a01 100644 --- a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/FeedReplicationTest.java +++ b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/FeedReplicationTest.java @@ -23,7 +23,6 @@ import org.apache.falcon.regression.core.bundle.Bundle; import org.apache.falcon.entity.v0.EntityType; import org.apache.falcon.entity.v0.feed.ActionType; import org.apache.falcon.entity.v0.feed.ClusterType; -import org.apache.falcon.entity.v0.feed.Feed; import org.apache.falcon.regression.core.helpers.ColoHelper; import org.apache.falcon.regression.core.response.InstancesResult; import org.apache.falcon.regression.core.util.AssertUtil; @@ -72,12 +71,11 @@ public class FeedReplicationTest extends BaseTestClass { private FileSystem cluster3FS = serverFS.get(2); private OozieClient cluster2OC = serverOC.get(1); private OozieClient cluster3OC = serverOC.get(2); - private String dateTemplate = "/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}"; private String baseTestDir = baseHDFSDir + "/FeedReplicationTest"; private String sourcePath = baseTestDir + "/source"; - private String feedDataLocation = baseTestDir + "/source" + dateTemplate; + private String feedDataLocation = baseTestDir + "/source" + MINUTE_DATE_PATTERN; private String targetPath = baseTestDir + "/target"; - private String targetDataLocation = targetPath + dateTemplate; + private String targetDataLocation = targetPath + MINUTE_DATE_PATTERN; private static final Logger LOGGER = Logger.getLogger(FeedReplicationTest.class); @BeforeMethod(alwaysRun = true) http://git-wip-us.apache.org/repos/asf/incubator-falcon/blob/be08a2e7/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/InstanceParamTest.java ---------------------------------------------------------------------- diff --git a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/InstanceParamTest.java b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/InstanceParamTest.java index edf3428..c56310e 100644 --- a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/InstanceParamTest.java +++ b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/InstanceParamTest.java @@ -58,8 +58,7 @@ public class InstanceParamTest extends BaseTestClass { */ private String baseTestHDFSDir = baseHDFSDir + "/InstanceParamTest"; - private String feedInputPath = baseTestHDFSDir + - "/testInputData/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}"; + private String feedInputPath = baseTestHDFSDir + "/testInputData" + MINUTE_DATE_PATTERN; private String aggregateWorkflowDir = baseTestHDFSDir + "/aggregator"; private String startTime; private String endTime; http://git-wip-us.apache.org/repos/asf/incubator-falcon/blob/be08a2e7/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/InstanceSummaryTest.java ---------------------------------------------------------------------- diff --git a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/InstanceSummaryTest.java b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/InstanceSummaryTest.java index 636da2c..75eccaf 100644 --- a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/InstanceSummaryTest.java +++ b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/InstanceSummaryTest.java @@ -61,8 +61,7 @@ import java.util.List; public class InstanceSummaryTest extends BaseTestClass { String baseTestHDFSDir = baseHDFSDir + "/InstanceSummaryTest"; - String feedInputPath = baseTestHDFSDir + - "/testInputData/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}"; + String feedInputPath = baseTestHDFSDir + "/testInputData" + MINUTE_DATE_PATTERN; String aggregateWorkflowDir = baseTestHDFSDir + "/aggregator"; String startTime; String endTime; http://git-wip-us.apache.org/repos/asf/incubator-falcon/blob/be08a2e7/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/NewRetryTest.java ---------------------------------------------------------------------- diff --git a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/NewRetryTest.java b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/NewRetryTest.java index 48bf185..c505a93 100644 --- a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/NewRetryTest.java +++ b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/NewRetryTest.java @@ -76,8 +76,8 @@ public class NewRetryTest extends BaseTestClass { DateTimeFormatter formatter = DateTimeFormat.forPattern("yyyy/MM/dd/HH/mm"); final private String baseTestDir = baseHDFSDir + "/NewRetryTest"; final private String aggregateWorkflowDir = baseTestDir + "/aggregator"; - final private String lateDir = baseTestDir + "/lateDataTest/testFolders/"; - final private String latePath = lateDir + "${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}"; + final private String lateDir = baseTestDir + "/lateDataTest/testFolders"; + final private String latePath = lateDir + MINUTE_DATE_PATTERN; private DateTime startDate; private DateTime endDate; http://git-wip-us.apache.org/repos/asf/incubator-falcon/blob/be08a2e7/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/NoOutputProcessTest.java ---------------------------------------------------------------------- diff --git a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/NoOutputProcessTest.java b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/NoOutputProcessTest.java index 2c30f83..638768d 100644 --- a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/NoOutputProcessTest.java +++ b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/NoOutputProcessTest.java @@ -57,7 +57,7 @@ public class NoOutputProcessTest extends BaseTestClass { FileSystem clusterFS = serverFS.get(0); OozieClient clusterOC = serverOC.get(0); String testDir = baseHDFSDir + "/NoOutputProcessTest"; - String inputPath = testDir + "/input/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}"; + String inputPath = testDir + "/input" + MINUTE_DATE_PATTERN; String aggregateWorkflowDir = testDir + "/aggregator"; private static final Logger logger = Logger.getLogger(NoOutputProcessTest.class); http://git-wip-us.apache.org/repos/asf/incubator-falcon/blob/be08a2e7/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/ProcessInstanceColoMixedTest.java ---------------------------------------------------------------------- diff --git a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/ProcessInstanceColoMixedTest.java b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/ProcessInstanceColoMixedTest.java index 25ef675..9c355a2 100644 --- a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/ProcessInstanceColoMixedTest.java +++ b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/ProcessInstanceColoMixedTest.java @@ -56,8 +56,7 @@ import java.util.List; public class ProcessInstanceColoMixedTest extends BaseTestClass { private final String baseTestHDFSDir = baseHDFSDir + "/ProcessInstanceColoMixedTest"; - private final String datePattern = "/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}/"; - private final String feedPath = baseTestHDFSDir + "/feed0%d" + datePattern; + private final String feedPath = baseTestHDFSDir + "/feed0%d" + MINUTE_DATE_PATTERN; private String aggregateWorkflowDir = baseTestHDFSDir + "/aggregator"; private ColoHelper cluster1 = servers.get(0); private ColoHelper cluster2 = servers.get(1); http://git-wip-us.apache.org/repos/asf/incubator-falcon/blob/be08a2e7/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/ProcessInstanceKillsTest.java ---------------------------------------------------------------------- diff --git a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/ProcessInstanceKillsTest.java b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/ProcessInstanceKillsTest.java index 33d6b07..ba7a8fd 100644 --- a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/ProcessInstanceKillsTest.java +++ b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/ProcessInstanceKillsTest.java @@ -56,9 +56,8 @@ public class ProcessInstanceKillsTest extends BaseTestClass { private String testDir = "/ProcessInstanceKillsTest"; private String baseTestHDFSDir = baseHDFSDir + testDir; private String aggregateWorkflowDir = baseTestHDFSDir + "/aggregator"; - private String datePattern = "/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}"; - private String feedInputPath = baseTestHDFSDir + "/input" + datePattern; - private String feedOutputPath = baseTestHDFSDir + "/output-data" + datePattern; + private String feedInputPath = baseTestHDFSDir + "/input" + MINUTE_DATE_PATTERN; + private String feedOutputPath = baseTestHDFSDir + "/output-data" + MINUTE_DATE_PATTERN; private static final Logger LOGGER = Logger.getLogger(ProcessInstanceKillsTest.class); private static final double TIMEOUT = 15; String processName; http://git-wip-us.apache.org/repos/asf/incubator-falcon/blob/be08a2e7/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/ProcessInstanceRerunTest.java ---------------------------------------------------------------------- diff --git a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/ProcessInstanceRerunTest.java b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/ProcessInstanceRerunTest.java index 3160c1d..8df7f1f 100644 --- a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/ProcessInstanceRerunTest.java +++ b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/ProcessInstanceRerunTest.java @@ -54,10 +54,9 @@ public class ProcessInstanceRerunTest extends BaseTestClass { private String baseTestDir = baseHDFSDir + "/ProcessInstanceRerunTest"; private String aggregateWorkflowDir = baseTestDir + "/aggregator"; - private String datePattern = "/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}"; - private String feedInputPath = baseTestDir + "/input" + datePattern; - private String feedOutputPath = baseTestDir + "/output-data" + datePattern; - private String feedInputTimedOutPath = baseTestDir + "/timedout" + datePattern; + private String feedInputPath = baseTestDir + "/input" + MINUTE_DATE_PATTERN; + private String feedOutputPath = baseTestDir + "/output-data" + MINUTE_DATE_PATTERN; + private String feedInputTimedOutPath = baseTestDir + "/timedout" + MINUTE_DATE_PATTERN; private ColoHelper cluster = servers.get(0); private FileSystem clusterFS = serverFS.get(0); private OozieClient clusterOC = serverOC.get(0); http://git-wip-us.apache.org/repos/asf/incubator-falcon/blob/be08a2e7/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/ProcessInstanceResumeTest.java ---------------------------------------------------------------------- diff --git a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/ProcessInstanceResumeTest.java b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/ProcessInstanceResumeTest.java index e324c26..fdb64af 100644 --- a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/ProcessInstanceResumeTest.java +++ b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/ProcessInstanceResumeTest.java @@ -53,10 +53,8 @@ public class ProcessInstanceResumeTest extends BaseTestClass { private ColoHelper cluster = servers.get(0); private FileSystem clusterFS = serverFS.get(0); private String baseTestHDFSDir = baseHDFSDir + "/ProcessInstanceResumeTest"; - private String feedInputPath = baseTestHDFSDir + - "/input/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}"; - private String feedOutputPath = - baseTestHDFSDir + "/output-data/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}"; + private String feedInputPath = baseTestHDFSDir + "/input" + MINUTE_DATE_PATTERN; + private String feedOutputPath = baseTestHDFSDir + "/output-data" + MINUTE_DATE_PATTERN; private String aggregateWorkflowDir = baseTestHDFSDir + "/aggregator"; private static final Logger LOGGER = Logger.getLogger(ProcessInstanceResumeTest.class); private static final double SCHEDULED = 15; http://git-wip-us.apache.org/repos/asf/incubator-falcon/blob/be08a2e7/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/ProcessInstanceRunningTest.java ---------------------------------------------------------------------- diff --git a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/ProcessInstanceRunningTest.java b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/ProcessInstanceRunningTest.java index 6bb466c..2b36d4d 100644 --- a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/ProcessInstanceRunningTest.java +++ b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/ProcessInstanceRunningTest.java @@ -57,10 +57,8 @@ public class ProcessInstanceRunningTest extends BaseTestClass { private FileSystem clusterFS = serverFS.get(0); private String baseTestHDFSDir = baseHDFSDir + "/ProcessInstanceRunningTest"; private String aggregateWorkflowDir = baseTestHDFSDir + "/aggregator"; - private String feedInputPath = baseTestHDFSDir + - "/input/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}"; - private String feedOutputPath = - baseTestHDFSDir + "/output-data/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}"; + private String feedInputPath = baseTestHDFSDir + "/input" + MINUTE_DATE_PATTERN; + private String feedOutputPath = baseTestHDFSDir + "/output-data" + MINUTE_DATE_PATTERN; private static final Logger LOGGER = Logger.getLogger(ProcessInstanceRunningTest.class); private static final double TIMEOUT = 15; http://git-wip-us.apache.org/repos/asf/incubator-falcon/blob/be08a2e7/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/ProcessInstanceStatusTest.java ---------------------------------------------------------------------- diff --git a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/ProcessInstanceStatusTest.java b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/ProcessInstanceStatusTest.java index 5e56037..cac8904 100644 --- a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/ProcessInstanceStatusTest.java +++ b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/ProcessInstanceStatusTest.java @@ -62,14 +62,12 @@ public class ProcessInstanceStatusTest extends BaseTestClass { private FileSystem clusterFS = serverFS.get(0); private String baseTestHDFSDir = baseHDFSDir + "/ProcessInstanceStatusTest"; private String aggregateWorkflowDir = baseTestHDFSDir + "/aggregator"; - private String feedInputPath = - baseTestHDFSDir + "/input/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}"; - private String feedOutputPath = - baseTestHDFSDir + "/output-data/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}"; - private String feedInputTimedOutPath = - baseTestHDFSDir + "/timedoutStatus/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}"; + private String feedInputPath = baseTestHDFSDir + "/input" + MINUTE_DATE_PATTERN; + private String feedOutputPath = baseTestHDFSDir + "/output-data" + MINUTE_DATE_PATTERN; + private String feedInputTimedOutPath = baseTestHDFSDir + "/timedoutStatus" + + MINUTE_DATE_PATTERN; private String feedOutputTimedOutPath = - baseTestHDFSDir + "/output-data/timedoutStatus/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}"; + baseTestHDFSDir + "/output-data/timedoutStatus" + MINUTE_DATE_PATTERN; private static final Logger LOGGER = Logger.getLogger(ProcessInstanceStatusTest.class); private static final double TIMEOUT = 15; http://git-wip-us.apache.org/repos/asf/incubator-falcon/blob/be08a2e7/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/ProcessInstanceSuspendTest.java ---------------------------------------------------------------------- diff --git a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/ProcessInstanceSuspendTest.java b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/ProcessInstanceSuspendTest.java index 02bd254..80da1ad 100644 --- a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/ProcessInstanceSuspendTest.java +++ b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/ProcessInstanceSuspendTest.java @@ -56,10 +56,8 @@ import java.util.List; public class ProcessInstanceSuspendTest extends BaseTestClass { private String baseTestHDFSDir = baseHDFSDir + "/ProcessInstanceSuspendTest"; - private String feedInputPath = baseTestHDFSDir + - "/input/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}"; - private String feedOutputPath = - baseTestHDFSDir + "/output-data/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}"; + private String feedInputPath = baseTestHDFSDir + MINUTE_DATE_PATTERN; + private String feedOutputPath = baseTestHDFSDir + "/output-data" + MINUTE_DATE_PATTERN; private String aggregateWorkflowDir = baseTestHDFSDir + "/aggregator"; private ColoHelper cluster = servers.get(0); private FileSystem clusterFS = serverFS.get(0); http://git-wip-us.apache.org/repos/asf/incubator-falcon/blob/be08a2e7/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/ProcessLibPathTest.java ---------------------------------------------------------------------- diff --git a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/ProcessLibPathTest.java b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/ProcessLibPathTest.java index fa02fdf..c2d8c9b 100644 --- a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/ProcessLibPathTest.java +++ b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/ProcessLibPathTest.java @@ -54,7 +54,6 @@ public class ProcessLibPathTest extends BaseTestClass { String testDir = baseHDFSDir + "/ProcessLibPath"; String testLibDir = testDir + "/TestLib"; private static final Logger logger = Logger.getLogger(ProcessLibPathTest.class); - String datePattern = "/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}"; String processName; String process; @@ -69,7 +68,7 @@ public class ProcessLibPathTest extends BaseTestClass { b = new Bundle(b, cluster); String startDate = "2010-01-01T22:00Z"; String endDate = "2010-01-02T03:00Z"; - b.setInputFeedDataPath(testDir + "/input" + datePattern); + b.setInputFeedDataPath(testDir + "/input" + MINUTE_DATE_PATTERN); String prefix = b.getFeedDataPathPrefix(); HadoopUtil.deleteDirIfExists(prefix.substring(1), clusterFS); List dataDates = TimeUtil.getMinuteDatesOnEitherSide(startDate, endDate, 20); @@ -82,11 +81,11 @@ public class ProcessLibPathTest extends BaseTestClass { bundles[0] = BundleUtil.readELBundle(); bundles[0] = new Bundle(bundles[0], cluster); bundles[0].generateUniqueBundle(); - bundles[0].setInputFeedDataPath(baseHDFSDir + datePattern); + bundles[0].setInputFeedDataPath(baseHDFSDir + MINUTE_DATE_PATTERN); bundles[0].setProcessValidity("2010-01-02T01:00Z", "2010-01-02T01:04Z"); bundles[0].setProcessPeriodicity(5, TimeUnit.minutes); bundles[0].setOutputFeedPeriodicity(5, TimeUnit.minutes); - bundles[0].setOutputFeedLocationData(baseHDFSDir + "/output-data" + datePattern); + bundles[0].setOutputFeedLocationData(baseHDFSDir + "/output-data" + MINUTE_DATE_PATTERN); bundles[0].setProcessConcurrency(1); bundles[0].setProcessLibPath(testLibDir); process = bundles[0].getProcessData(); http://git-wip-us.apache.org/repos/asf/incubator-falcon/blob/be08a2e7/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/lineage/LineageApiProcessInstanceTest.java ---------------------------------------------------------------------- diff --git a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/lineage/LineageApiProcessInstanceTest.java b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/lineage/LineageApiProcessInstanceTest.java index 838c6d7..5868882 100644 --- a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/lineage/LineageApiProcessInstanceTest.java +++ b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/lineage/LineageApiProcessInstanceTest.java @@ -60,9 +60,8 @@ public class LineageApiProcessInstanceTest extends BaseTestClass { String baseTestHDFSDir = baseHDFSDir + "/LineageApiInstanceTest"; String aggregateWorkflowDir = baseTestHDFSDir + "/aggregator"; String feedInputPrefix = baseTestHDFSDir + "/input"; - String feedInputPath = feedInputPrefix + "/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}"; - String feedOutputPath = - baseTestHDFSDir + "/output-data/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}"; + String feedInputPath = feedInputPrefix + MINUTE_DATE_PATTERN; + String feedOutputPath = baseTestHDFSDir + "/output-data" + MINUTE_DATE_PATTERN; String processName; String inputFeedName; String outputFeedName; http://git-wip-us.apache.org/repos/asf/incubator-falcon/blob/be08a2e7/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/lineage/LineageApiTest.java ---------------------------------------------------------------------- diff --git a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/lineage/LineageApiTest.java b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/lineage/LineageApiTest.java index 37aa98c..0c6ecae 100644 --- a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/lineage/LineageApiTest.java +++ b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/lineage/LineageApiTest.java @@ -57,7 +57,6 @@ import java.util.TreeMap; @Test(groups = "lineage-rest") public class LineageApiTest extends BaseTestClass { - private static final String datePattern = "${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}"; private static final Logger logger = Logger.getLogger(LineageApiTest.class); private static final String testName = "LineageApiTest"; private static final String testTag = @@ -100,7 +99,7 @@ public class LineageApiTest extends BaseTestClass { inputMerlin.setTags(testTag); inputFeeds = generateFeeds(numInputFeeds, inputMerlin, Generator.getNameGenerator("infeed", inputMerlin.getName()), - Generator.getHadoopPathGenerator(feedInputPath, datePattern)); + Generator.getHadoopPathGenerator(feedInputPath, MINUTE_DATE_PATTERN)); for (FeedMerlin feed : inputFeeds) { AssertUtil.assertSucceeded(prism.getFeedHelper().submitEntity(Util.URLS.SUBMIT_URL, feed.toString())); @@ -110,7 +109,7 @@ public class LineageApiTest extends BaseTestClass { outputMerlin.setTags(testTag); outputFeeds = generateFeeds(numOutputFeeds, outputMerlin, Generator.getNameGenerator("outfeed", outputMerlin.getName()), - Generator.getHadoopPathGenerator(feedOutputPath, datePattern)); + Generator.getHadoopPathGenerator(feedOutputPath, MINUTE_DATE_PATTERN)); for (FeedMerlin feed : outputFeeds) { AssertUtil.assertSucceeded(prism.getFeedHelper().submitEntity(Util.URLS.SUBMIT_URL, feed.toString())); http://git-wip-us.apache.org/repos/asf/incubator-falcon/blob/be08a2e7/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/EntityDryRunTest.java ---------------------------------------------------------------------- diff --git a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/EntityDryRunTest.java b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/EntityDryRunTest.java index 0b06823..72d03cd 100644 --- a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/EntityDryRunTest.java +++ b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/EntityDryRunTest.java @@ -51,10 +51,8 @@ public class EntityDryRunTest extends BaseTestClass { private ColoHelper cluster = servers.get(0); private FileSystem clusterFS = serverFS.get(0); private String baseTestHDFSDir = baseHDFSDir + "/EntityDryRunTest"; - private String feedInputPath = baseTestHDFSDir + - "/input/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}"; - private String feedOutputPath = - baseTestHDFSDir + "/output-data/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}"; + private String feedInputPath = baseTestHDFSDir + "/input" + MINUTE_DATE_PATTERN; + private String feedOutputPath = baseTestHDFSDir + "/output-data" + MINUTE_DATE_PATTERN; private String aggregateWorkflowDir = baseTestHDFSDir + "/aggregator"; private static final Logger LOGGER = Logger.getLogger(EntityDryRunTest.class); http://git-wip-us.apache.org/repos/asf/incubator-falcon/blob/be08a2e7/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/FeedDelayParallelTimeoutTest.java ---------------------------------------------------------------------- diff --git a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/FeedDelayParallelTimeoutTest.java b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/FeedDelayParallelTimeoutTest.java index 564637c..c234f0d 100644 --- a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/FeedDelayParallelTimeoutTest.java +++ b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/FeedDelayParallelTimeoutTest.java @@ -46,7 +46,7 @@ public class FeedDelayParallelTimeoutTest extends BaseTestClass { ColoHelper cluster2 = servers.get(1); String baseTestDir = baseHDFSDir + "/FeedDelayParallelTimeoutTest"; - String feedInputPath = baseTestDir + "/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}/"; + String feedInputPath = baseTestDir + MINUTE_DATE_PATTERN; String aggregateWorkflowDir = baseTestDir + "/aggregator"; private static final Logger logger = Logger.getLogger(FeedDelayParallelTimeoutTest.class); http://git-wip-us.apache.org/repos/asf/incubator-falcon/blob/be08a2e7/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/NewPrismProcessUpdateTest.java ---------------------------------------------------------------------- diff --git a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/NewPrismProcessUpdateTest.java b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/NewPrismProcessUpdateTest.java index 884862b..9198901 100644 --- a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/NewPrismProcessUpdateTest.java +++ b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/NewPrismProcessUpdateTest.java @@ -73,7 +73,7 @@ import java.util.Random; public class NewPrismProcessUpdateTest extends BaseTestClass { private String baseTestDir = baseHDFSDir + "/NewPrismProcessUpdateTest"; - private String inputFeedPath = baseTestDir + "/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}"; + private String inputFeedPath = baseTestDir + MINUTE_DATE_PATTERN; private String workflowPath = baseTestDir + "/falcon-oozie-wf"; private String workflowPath2 = baseTestDir + "/falcon-oozie-wf2"; private String aggregatorPath = baseTestDir + "/aggregator"; http://git-wip-us.apache.org/repos/asf/incubator-falcon/blob/be08a2e7/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/PrismFeedDeleteTest.java ---------------------------------------------------------------------- diff --git a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/PrismFeedDeleteTest.java b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/PrismFeedDeleteTest.java index b1e78d0..9e3ed12 100644 --- a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/PrismFeedDeleteTest.java +++ b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/PrismFeedDeleteTest.java @@ -399,15 +399,13 @@ public class PrismFeedDeleteTest extends BaseTestClass { .setFeedCluster(feed, XmlUtil.createValidity(startTimeServer1, "2099-10-01T12:10Z"), XmlUtil.createRtention("days(10000)", ActionType.DELETE), Util.readEntityName(bundles[0].getClusters().get(0)), ClusterType.SOURCE, - "${cluster.colo}", - baseHDFSDir + "/localDC/rc/billing/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}"); + "${cluster.colo}", baseHDFSDir + "/localDC/rc/billing" + MINUTE_DATE_PATTERN); feed = InstanceUtil .setFeedCluster(feed, XmlUtil.createValidity(startTimeServer2, "2099-10-01T12:25Z"), XmlUtil.createRtention("days(10000)", ActionType.DELETE), Util.readEntityName(bundles[1].getClusters().get(0)), ClusterType.TARGET, null, - baseHDFSDir + - "/clusterPath/localDC/rc/billing/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}"); + baseHDFSDir + "/clusterPath/localDC/rc/billing" + MINUTE_DATE_PATTERN); Util.shutDownService(cluster1.getFeedHelper()); @@ -836,15 +834,13 @@ public class PrismFeedDeleteTest extends BaseTestClass { .setFeedCluster(feed, XmlUtil.createValidity(startTimeServer1, "2099-10-01T12:10Z"), XmlUtil.createRtention("days(10000)", ActionType.DELETE), Util.readEntityName(bundles[0].getClusters().get(0)), ClusterType.SOURCE, - "${cluster.colo}", - baseHDFSDir + "/localDC/rc/billing/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}"); + "${cluster.colo}", baseHDFSDir + "/localDC/rc/billing" + MINUTE_DATE_PATTERN); feed = InstanceUtil .setFeedCluster(feed, XmlUtil.createValidity(startTimeServer2, "2099-10-01T12:25Z"), XmlUtil.createRtention("days(10000)", ActionType.DELETE), Util.readEntityName(bundles[1].getClusters().get(0)), ClusterType.TARGET, null, - baseHDFSDir + - "/clusterPath/localDC/rc/billing/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}"); + baseHDFSDir + "/clusterPath/localDC/rc/billing" + MINUTE_DATE_PATTERN); logger.info("feed: " + Util.prettyPrintXml(feed)); @@ -939,14 +935,12 @@ public class PrismFeedDeleteTest extends BaseTestClass { .setFeedCluster(feed, XmlUtil.createValidity(startTimeServer1, "2099-10-01T12:10Z"), XmlUtil.createRtention("days(10000)", ActionType.DELETE), Util.readEntityName(bundles[0].getClusters().get(0)), ClusterType.SOURCE, - "${cluster.colo}", - baseHDFSDir + "/localDC/rc/billing/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}"); + "${cluster.colo}", baseHDFSDir + "/localDC/rc/billing" + MINUTE_DATE_PATTERN); feed = InstanceUtil .setFeedCluster(feed, XmlUtil.createValidity(startTimeServer2, "2099-10-01T12:25Z"), XmlUtil.createRtention("days(10000)", ActionType.DELETE), Util.readEntityName(bundles[1].getClusters().get(0)), ClusterType.TARGET, null, - baseHDFSDir + "/clusterPath/localDC/rc/billing/${YEAR}/${MONTH}/${DAY}/${HOUR}/$" + - "{MINUTE}"); + baseHDFSDir + "/clusterPath/localDC/rc/billing" + MINUTE_DATE_PATTERN); logger.info("feed: " + Util.prettyPrintXml(feed)); http://git-wip-us.apache.org/repos/asf/incubator-falcon/blob/be08a2e7/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/PrismFeedLateReplicationTest.java ---------------------------------------------------------------------- diff --git a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/PrismFeedLateReplicationTest.java b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/PrismFeedLateReplicationTest.java index 71abd95..8a14229 100644 --- a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/PrismFeedLateReplicationTest.java +++ b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/PrismFeedLateReplicationTest.java @@ -56,8 +56,7 @@ public class PrismFeedLateReplicationTest extends BaseTestClass { FileSystem cluster2FS = serverFS.get(1); FileSystem cluster3FS = serverFS.get(2); private String baseTestDir = baseHDFSDir + "/PrismFeedLateReplicationTest"; - private String inputPath = - baseTestDir + "/input-data/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}/"; + private String inputPath = baseTestDir + "/input-data" + MINUTE_DATE_PATTERN; private String aggregateWorkflowDir = baseTestDir + "/aggregator"; private static final Logger logger = Logger.getLogger(PrismFeedLateReplicationTest.class); http://git-wip-us.apache.org/repos/asf/incubator-falcon/blob/be08a2e7/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/PrismFeedReplicationPartitionExpTest.java ---------------------------------------------------------------------- diff --git a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/PrismFeedReplicationPartitionExpTest.java b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/PrismFeedReplicationPartitionExpTest.java index ee7052e..2d67583 100755 --- a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/PrismFeedReplicationPartitionExpTest.java +++ b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/PrismFeedReplicationPartitionExpTest.java @@ -72,7 +72,6 @@ public class PrismFeedReplicationPartitionExpTest extends BaseTestClass { private String testDirWithDate = testBaseDir1 + testDate; private String testDirWithDate_sourcetarget = testBaseDir4 + testDate; private String testDirWithDate_source1 = testBaseDir_server1source + testDate; - private String dateTemplate = "/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}"; private String testFile1 = OSUtil.RESOURCES + OSUtil.getPath("ReplicationResources", "feed-s4Replication.xml"); private String testFile2 = OSUtil.RESOURCES + OSUtil.getPath("ReplicationResources", "id.pig"); @@ -216,13 +215,13 @@ public class PrismFeedReplicationPartitionExpTest extends BaseTestClass { .setFeedCluster(feed, XmlUtil.createValidity(startTimeUA1, "2012-10-01T12:10Z"), XmlUtil.createRtention("days(1000000)", ActionType.DELETE), Util.readEntityName(bundles[0].getClusters().get(0)), ClusterType.SOURCE, "", - testBaseDir1 + dateTemplate); + testBaseDir1 + MINUTE_DATE_PATTERN); feed = InstanceUtil .setFeedCluster(feed, XmlUtil.createValidity(startTimeUA2, "2012-10-01T12:25Z"), XmlUtil.createRtention("days(1000000)", ActionType.DELETE), Util.readEntityName(bundles[1].getClusters().get(0)), ClusterType.TARGET, "", - testBaseDir2 + dateTemplate); + testBaseDir2 + MINUTE_DATE_PATTERN); feed = InstanceUtil.setFeedCluster(feed, XmlUtil.createValidity("2012-10-01T12:00Z", "2099-01-01T00:00Z"), @@ -269,13 +268,13 @@ public class PrismFeedReplicationPartitionExpTest extends BaseTestClass { .setFeedCluster(feed, XmlUtil.createValidity(startTimeUA2, "2099-10-01T12:25Z"), XmlUtil.createRtention("days(100000)", ActionType.DELETE), Util.readEntityName(bundles[1].getClusters().get(0)), ClusterType.TARGET, null, - testBaseDir2 + dateTemplate); + testBaseDir2 + MINUTE_DATE_PATTERN); feed = InstanceUtil .setFeedCluster(feed, XmlUtil.createValidity("2012-10-01T12:00Z", "2099-01-01T00:00Z"), XmlUtil.createRtention("days(100000)", ActionType.DELETE), Util.readEntityName(bundles[2].getClusters().get(0)), ClusterType.SOURCE, - "${cluster.colo}", testBaseDir1 + dateTemplate); + "${cluster.colo}", testBaseDir1 + MINUTE_DATE_PATTERN); logger.info("feed: " + Util.prettyPrintXml(feed)); @@ -364,13 +363,13 @@ public class PrismFeedReplicationPartitionExpTest extends BaseTestClass { .setFeedCluster(feed, XmlUtil.createValidity(startTimeUA2, "2099-10-01T12:25Z"), XmlUtil.createRtention("days(1000000)", ActionType.DELETE), Util.readEntityName(bundles[1].getClusters().get(0)), ClusterType.TARGET, - "${cluster.colo}", testBaseDir2 + dateTemplate); + "${cluster.colo}", testBaseDir2 + MINUTE_DATE_PATTERN); feed = InstanceUtil.setFeedCluster(feed, XmlUtil.createValidity("2012-10-01T12:00Z", "2099-01-01T00:00Z"), XmlUtil.createRtention("days(1000000)", ActionType.DELETE), Util.readEntityName(bundles[2].getClusters().get(0)), ClusterType.SOURCE, null, - testBaseDir1 + dateTemplate); + testBaseDir1 + MINUTE_DATE_PATTERN); logger.info("feed: " + Util.prettyPrintXml(feed)); @@ -440,7 +439,7 @@ public class PrismFeedReplicationPartitionExpTest extends BaseTestClass { String feed = bundles[0].getDataSets().get(0); - feed = InstanceUtil.setFeedFilePath(feed, testBaseDir3 + dateTemplate); + feed = InstanceUtil.setFeedFilePath(feed, testBaseDir3 + MINUTE_DATE_PATTERN); feed = InstanceUtil.setFeedCluster(feed, XmlUtil.createValidity("2012-10-01T12:00Z", "2010-01-01T00:00Z"), @@ -547,14 +546,14 @@ public class PrismFeedReplicationPartitionExpTest extends BaseTestClass { .setFeedCluster(feed, XmlUtil.createValidity(startTimeUA1, "2012-10-01T12:10Z"), XmlUtil.createRtention("days(1000000)", ActionType.DELETE), Util.readEntityName(bundles[0].getClusters().get(0)), ClusterType.SOURCE, null, - testBaseDir1 + dateTemplate); + testBaseDir1 + MINUTE_DATE_PATTERN); feed = InstanceUtil .setFeedCluster(feed, XmlUtil.createValidity(startTimeUA2, "2012-10-01T12:25Z"), XmlUtil.createRtention("days(1000000)", ActionType.DELETE), Util.readEntityName(bundles[1].getClusters().get(0)), ClusterType.TARGET, "${cluster.colo}", - testBaseDir2 + dateTemplate); + testBaseDir2 + MINUTE_DATE_PATTERN); feed = InstanceUtil.setFeedCluster(feed, XmlUtil.createValidity("2012-10-01T12:00Z", "2099-01-01T00:00Z"), @@ -594,7 +593,7 @@ public class PrismFeedReplicationPartitionExpTest extends BaseTestClass { String feed = bundles[0].getDataSets().get(0); feed = InstanceUtil.setFeedFilePath(feed, - testBaseDir1 + dateTemplate); + testBaseDir1 + MINUTE_DATE_PATTERN); feed = InstanceUtil.setFeedCluster(feed, XmlUtil.createValidity("2012-10-01T12:00Z", "2010-01-01T00:00Z"), XmlUtil.createRtention("days(10000000)", ActionType.DELETE), null, @@ -604,13 +603,13 @@ public class PrismFeedReplicationPartitionExpTest extends BaseTestClass { .setFeedCluster(feed, XmlUtil.createValidity(startTimeUA1, "2012-10-01T12:11Z"), XmlUtil.createRtention("days(10000000)", ActionType.DELETE), Util.readEntityName(bundles[0].getClusters().get(0)), ClusterType.TARGET, null, - testBaseDir1 + "/ua1" + dateTemplate); + testBaseDir1 + "/ua1" + MINUTE_DATE_PATTERN); feed = InstanceUtil .setFeedCluster(feed, XmlUtil.createValidity(startTimeUA2, "2012-10-01T12:26Z"), XmlUtil.createRtention("days(10000000)", ActionType.DELETE), Util.readEntityName(bundles[1].getClusters().get(0)), ClusterType.TARGET, null, - testBaseDir1 + "/ua2" + dateTemplate); + testBaseDir1 + "/ua2" + MINUTE_DATE_PATTERN); feed = InstanceUtil.setFeedCluster(feed, XmlUtil.createValidity("2012-10-01T12:00Z", "2099-01-01T00:00Z"), @@ -705,19 +704,19 @@ public class PrismFeedReplicationPartitionExpTest extends BaseTestClass { XmlUtil.createRtention("days(1000000)", ActionType.DELETE), Util.readEntityName(bundles[0].getClusters().get(0)), ClusterType.SOURCE, "${cluster.colo}", - testBaseDir_server1source + dateTemplate); + testBaseDir_server1source + MINUTE_DATE_PATTERN); feed = InstanceUtil .setFeedCluster(feed, XmlUtil.createValidity(startTimeUA2, "2099-10-01T12:25Z"), XmlUtil.createRtention("days(1000000)", ActionType.DELETE), Util.readEntityName(bundles[1].getClusters().get(0)), ClusterType.TARGET, null, - testBaseDir2 + "/replicated" + dateTemplate); + testBaseDir2 + "/replicated" + MINUTE_DATE_PATTERN); feed = InstanceUtil.setFeedCluster(feed, XmlUtil.createValidity("2012-10-01T12:00Z", "2099-01-01T00:00Z"), XmlUtil.createRtention("days(1000000)", ActionType.DELETE), Util.readEntityName(bundles[2].getClusters().get(0)), ClusterType.SOURCE, - "${cluster.colo}", testBaseDir1 + dateTemplate); + "${cluster.colo}", testBaseDir1 + MINUTE_DATE_PATTERN); logger.info("feed: " + Util.prettyPrintXml(feed)); @@ -777,7 +776,7 @@ public class PrismFeedReplicationPartitionExpTest extends BaseTestClass { String startTimeUA2 = "2012-10-01T12:10Z"; String feed = bundles[0].getDataSets().get(0); - feed = InstanceUtil.setFeedFilePath(feed, testBaseDir1 + dateTemplate); + feed = InstanceUtil.setFeedFilePath(feed, testBaseDir1 + MINUTE_DATE_PATTERN); feed = InstanceUtil.setFeedCluster(feed, XmlUtil.createValidity("2012-10-01T12:00Z", "2010-01-01T00:00Z"), XmlUtil.createRtention("days(1000000)", ActionType.DELETE), null, @@ -787,19 +786,19 @@ public class PrismFeedReplicationPartitionExpTest extends BaseTestClass { .setFeedCluster(feed, XmlUtil.createValidity(startTimeUA1, "2099-10-01T12:10Z"), XmlUtil.createRtention("days(1000000)", ActionType.DELETE), Util.readEntityName(bundles[0].getClusters().get(0)), ClusterType.TARGET, - "${cluster.colo}", testBaseDir1 + "/ua1" + dateTemplate + "/"); + "${cluster.colo}", testBaseDir1 + "/ua1" + MINUTE_DATE_PATTERN + "/"); feed = InstanceUtil .setFeedCluster(feed, XmlUtil.createValidity(startTimeUA2, "2099-10-01T12:25Z"), XmlUtil.createRtention("days(1000000)", ActionType.DELETE), Util.readEntityName(bundles[1].getClusters().get(0)), ClusterType.TARGET, - "${cluster.colo}", testBaseDir1 + "/ua2" + dateTemplate + "/"); + "${cluster.colo}", testBaseDir1 + "/ua2" + MINUTE_DATE_PATTERN + "/"); feed = InstanceUtil.setFeedCluster(feed, XmlUtil.createValidity("2012-10-01T12:00Z", "2099-01-01T00:00Z") , XmlUtil.createRtention("days(1000000)", ActionType.DELETE), Util.readEntityName(bundles[2].getClusters().get(0)), ClusterType.SOURCE, - "${cluster.colo}", testBaseDir4 + dateTemplate + "/"); + "${cluster.colo}", testBaseDir4 + MINUTE_DATE_PATTERN + "/"); logger.info("feed: " + Util.prettyPrintXml(feed)); @@ -876,13 +875,13 @@ public class PrismFeedReplicationPartitionExpTest extends BaseTestClass { .setFeedCluster(feed, XmlUtil.createValidity(startTimeUA1, "2012-10-01T12:10Z"), XmlUtil.createRtention("days(1000000)", ActionType.DELETE), Util.readEntityName(bundles[0].getClusters().get(0)), ClusterType.SOURCE, "", - testBaseDir1 + dateTemplate); + testBaseDir1 + MINUTE_DATE_PATTERN); feed = InstanceUtil .setFeedCluster(feed, XmlUtil.createValidity(startTimeUA2, "2012-10-01T12:25Z"), XmlUtil.createRtention("days(1000000)", ActionType.DELETE), Util.readEntityName(bundles[2].getClusters().get(0)), ClusterType.TARGET, "", - testBaseDir2 + dateTemplate); + testBaseDir2 + MINUTE_DATE_PATTERN); feed = InstanceUtil.setFeedCluster(feed, XmlUtil.createValidity("2012-10-01T12:00Z", "2099-01-01T00:00Z"), http://git-wip-us.apache.org/repos/asf/incubator-falcon/blob/be08a2e7/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/PrismFeedReplicationUpdateTest.java ---------------------------------------------------------------------- diff --git a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/PrismFeedReplicationUpdateTest.java b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/PrismFeedReplicationUpdateTest.java index 6aebbd7..a4c27d7 100644 --- a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/PrismFeedReplicationUpdateTest.java +++ b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/PrismFeedReplicationUpdateTest.java @@ -59,10 +59,9 @@ public class PrismFeedReplicationUpdateTest extends BaseTestClass { String cluster2Colo = cluster2.getClusterHelper().getColoName(); String cluster3Colo = cluster3.getClusterHelper().getColoName(); private final String baseTestDir = baseHDFSDir + "/PrismFeedReplicationUpdateTest"; - private final String inputPath = - baseTestDir + "/input-data/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}"; - private String alternativeInputPath = - baseTestDir + "/newFeedPath/input-data/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}"; + private final String inputPath = baseTestDir + "/input-data" + MINUTE_DATE_PATTERN; + private String alternativeInputPath = baseTestDir + "/newFeedPath/input-data" + + MINUTE_DATE_PATTERN; private String aggregateWorkflowDir = baseTestDir + "/aggregator"; private static final Logger logger = Logger.getLogger(PrismFeedReplicationUpdateTest.class); @@ -213,10 +212,8 @@ public class PrismFeedReplicationUpdateTest extends BaseTestClass { ClusterType.SOURCE, null); //set new feed input data - feed01 = Util.setFeedPathValue(feed01, - baseHDFSDir + "/feed01/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}/"); - feed02 = Util.setFeedPathValue(feed02, - baseHDFSDir + "/feed02/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}/"); + feed01 = Util.setFeedPathValue(feed01, baseHDFSDir + "/feed01" + MINUTE_DATE_PATTERN); + feed02 = Util.setFeedPathValue(feed02, baseHDFSDir + "/feed02" + MINUTE_DATE_PATTERN); //generate data in both the colos ua1 and ua3 String prefix = InstanceUtil.getFeedPrefix(feed01); http://git-wip-us.apache.org/repos/asf/incubator-falcon/blob/be08a2e7/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/PrismFeedSnSTest.java ---------------------------------------------------------------------- diff --git a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/PrismFeedSnSTest.java b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/PrismFeedSnSTest.java index 15a2c3c..71f3751 100644 --- a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/PrismFeedSnSTest.java +++ b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/PrismFeedSnSTest.java @@ -418,12 +418,12 @@ public class PrismFeedSnSTest extends BaseTestClass { .setFeedCluster(feed, XmlUtil.createValidity(startTimeUA1, "2099-10-01T12:10Z"), XmlUtil.createRtention("days(10000)", ActionType.DELETE), Util.readEntityName(clust1), ClusterType.SOURCE, "${cluster.colo}", - baseHDFSDir + "/localDC/rc/billing/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}"); + baseHDFSDir + "/localDC/rc/billing" + MINUTE_DATE_PATTERN); feed = InstanceUtil .setFeedCluster(feed, XmlUtil.createValidity(startTimeUA2, "2099-10-01T12:25Z"), XmlUtil.createRtention("days(10000)", ActionType.DELETE), Util.readEntityName(clust2), ClusterType.TARGET, null, baseHDFSDir + - "/clusterPath/localDC/rc/billing/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}"); + "/clusterPath/localDC/rc/billing" + MINUTE_DATE_PATTERN); logger.info("feed: " + Util.prettyPrintXml(feed)); Util.shutDownService(cluster1.getFeedHelper()); http://git-wip-us.apache.org/repos/asf/incubator-falcon/blob/be08a2e7/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/PrismFeedUpdateTest.java ---------------------------------------------------------------------- diff --git a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/PrismFeedUpdateTest.java b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/PrismFeedUpdateTest.java index c0e1617..1225525 100644 --- a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/PrismFeedUpdateTest.java +++ b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/PrismFeedUpdateTest.java @@ -66,8 +66,7 @@ public class PrismFeedUpdateTest extends BaseTestClass { public final String cluster1colo = cluster1.getClusterHelper().getColoName(); public final String cluster2colo = cluster2.getClusterHelper().getColoName(); private static final Logger logger = Logger.getLogger(PrismFeedUpdateTest.class); - String feedInputTimedOutPath = - baseTestDir + "/timedout/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}"; + String feedInputTimedOutPath = baseTestDir + "/timedout" + MINUTE_DATE_PATTERN; @BeforeClass(alwaysRun = true) public void uploadWorkflow() throws Exception { @@ -131,8 +130,7 @@ public class PrismFeedUpdateTest extends BaseTestClass { ClusterType.SOURCE, null); //set new feed input data - feed01 = Util.setFeedPathValue(feed01, - baseTestDir + "/feed01/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}/"); + feed01 = Util.setFeedPathValue(feed01, baseTestDir + "/feed01" + MINUTE_DATE_PATTERN); //generate data in both the colos cluster1colo and cluster2colo String prefix = InstanceUtil.getFeedPrefix(feed01); http://git-wip-us.apache.org/repos/asf/incubator-falcon/blob/be08a2e7/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/RescheduleKilledProcessTest.java ---------------------------------------------------------------------- diff --git a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/RescheduleKilledProcessTest.java b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/RescheduleKilledProcessTest.java index 468692a..139b472 100644 --- a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/RescheduleKilledProcessTest.java +++ b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/RescheduleKilledProcessTest.java @@ -113,8 +113,7 @@ public class RescheduleKilledProcessTest extends BaseTestClass { bundles[0].setProcessValidity(TimeUtil.getTimeWrtSystemTime(-11), TimeUtil.getTimeWrtSystemTime(6)); - bundles[0].setInputFeedDataPath( - baseHDFSDir + "/rawLogs/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}"); + bundles[0].setInputFeedDataPath(baseHDFSDir + "/rawLogs" + MINUTE_DATE_PATTERN); logger.info("process: " + Util.prettyPrintXml(bundles[0].getProcessData())); http://git-wip-us.apache.org/repos/asf/incubator-falcon/blob/be08a2e7/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/RescheduleProcessInFinalStatesTest.java ---------------------------------------------------------------------- diff --git a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/RescheduleProcessInFinalStatesTest.java b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/RescheduleProcessInFinalStatesTest.java index 7109829..3bba852 100644 --- a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/RescheduleProcessInFinalStatesTest.java +++ b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/RescheduleProcessInFinalStatesTest.java @@ -56,7 +56,7 @@ public class RescheduleProcessInFinalStatesTest extends BaseTestClass { FileSystem clusterFS = serverFS.get(0); String baseTestDir = baseHDFSDir + "/RescheduleProcessInFinalStates"; String aggregateWorkflowDir = baseTestDir + "/aggregator"; - String inputPath = baseTestDir + "/input/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}"; + String inputPath = baseTestDir + "/input" + MINUTE_DATE_PATTERN; private static final Logger logger = Logger.getLogger(RescheduleProcessInFinalStatesTest.class); @BeforeClass(alwaysRun = true) @@ -92,8 +92,7 @@ public class RescheduleProcessInFinalStatesTest extends BaseTestClass { bundles[0].setProcessValidity("2010-01-02T01:00Z", "2010-01-02T01:15Z"); bundles[0].setProcessPeriodicity(5, TimeUnit.minutes); bundles[0].setOutputFeedPeriodicity(5, TimeUnit.minutes); - bundles[0].setOutputFeedLocationData( - baseTestDir + "/output-data/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}"); + bundles[0].setOutputFeedLocationData(baseTestDir + "/output-data" + MINUTE_DATE_PATTERN); bundles[0].setProcessConcurrency(6); bundles[0].setProcessWorkflow(aggregateWorkflowDir); bundles[0].submitFeedsScheduleProcess(prism); http://git-wip-us.apache.org/repos/asf/incubator-falcon/blob/be08a2e7/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/UpdateAtSpecificTimeTest.java ---------------------------------------------------------------------- diff --git a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/UpdateAtSpecificTimeTest.java b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/UpdateAtSpecificTimeTest.java index f8a9fd6..cd7d7bd 100644 --- a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/UpdateAtSpecificTimeTest.java +++ b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/prism/UpdateAtSpecificTimeTest.java @@ -68,7 +68,6 @@ public class UpdateAtSpecificTimeTest extends BaseTestClass { ColoHelper cluster2 = servers.get(1); ColoHelper cluster3 = servers.get(2); FileSystem cluster2FS = serverFS.get(1); - private String dateTemplate = "/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}"; private final String baseTestDir = baseHDFSDir + "/UpdateAtSpecificTimeTest-data"; String aggregateWorkflowDir = baseHDFSDir + "/aggregator"; @@ -440,7 +439,7 @@ public class UpdateAtSpecificTimeTest extends BaseTestClass { feed = InstanceUtil.setFeedCluster(feed, XmlUtil.createValidity(startTime, endTime), XmlUtil.createRtention("days(100000)", ActionType.DELETE), Util.readEntityName(processBundle.getClusters().get(0)), ClusterType.SOURCE, - null, baseTestDir + "/replication" + dateTemplate); + null, baseTestDir + "/replication" + MINUTE_DATE_PATTERN); ServiceResponse r = prism.getClusterHelper().submitEntity(Util.URLS.SUBMIT_URL, processBundle.getClusters().get(0)); @@ -585,7 +584,7 @@ public class UpdateAtSpecificTimeTest extends BaseTestClass { ("2012-10-01T12:10Z", "2099-10-01T12:10Z"), XmlUtil.createRtention("days(1000000)", ActionType.DELETE), Util.readEntityName(b.getClusters().get(0)), ClusterType.SOURCE, "", - "/someTestPath" + dateTemplate); + "/someTestPath" + MINUTE_DATE_PATTERN); ServiceResponse r = prism.getClusterHelper().submitEntity(Util.URLS .SUBMIT_URL, b.getClusters().get(0)); @@ -617,12 +616,12 @@ public class UpdateAtSpecificTimeTest extends BaseTestClass { XmlUtil.createValidity(startTimeCluster_target, "2099-10-01T12:25Z"), XmlUtil.createRtention("days(100000)", ActionType.DELETE), Util.readEntityName(bundles[0].getClusters().get(0)), ClusterType.TARGET, - null, testDataDir + dateTemplate); + null, testDataDir + MINUTE_DATE_PATTERN); feed = InstanceUtil.setFeedCluster(feed, XmlUtil.createValidity(startTimeCluster_source, "2099-01-01T00:00Z"), XmlUtil.createRtention("days(100000)", ActionType.DELETE), Util.readEntityName(bundles[1].getClusters().get(0)), ClusterType.SOURCE, - null, testDataDir + dateTemplate); + null, testDataDir + MINUTE_DATE_PATTERN); //submit clusters Bundle.submitCluster(bundles[0], bundles[1], bundles[2]); http://git-wip-us.apache.org/repos/asf/incubator-falcon/blob/be08a2e7/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/ui/LineageGraphTest.java ---------------------------------------------------------------------- diff --git a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/ui/LineageGraphTest.java b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/ui/LineageGraphTest.java index 697d8bd..7a69e58 100644 --- a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/ui/LineageGraphTest.java +++ b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/ui/LineageGraphTest.java @@ -62,8 +62,7 @@ public class LineageGraphTest extends BaseUITestClass { private String baseTestDir = baseHDFSDir + "/LineageGraphTest"; private String aggregateWorkflowDir = baseTestDir + "/aggregator"; private static final Logger logger = Logger.getLogger(LineageGraphTest.class); - String datePattern = "/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}"; - String feedInputPath = baseTestDir + "/input" + datePattern; + String feedInputPath = baseTestDir + "/input" + MINUTE_DATE_PATTERN; private FileSystem clusterFS = serverFS.get(0); private OozieClient clusterOC = serverOC.get(0); private String processName = null; http://git-wip-us.apache.org/repos/asf/incubator-falcon/blob/be08a2e7/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/ui/ProcessUITest.java ---------------------------------------------------------------------- diff --git a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/ui/ProcessUITest.java b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/ui/ProcessUITest.java index c82786f..110e2e9 100644 --- a/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/ui/ProcessUITest.java +++ b/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/ui/ProcessUITest.java @@ -69,7 +69,6 @@ public class ProcessUITest extends BaseUITestClass { private String baseTestDir = baseHDFSDir + "/TestProcessUI"; private String aggregateWorkflowDir = baseTestDir + "/aggregator"; private static final Logger logger = Logger.getLogger(ProcessUITest.class); - String datePattern = "/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}"; String feedInputPath = baseTestDir + "/input"; final String feedOutputPath = baseTestDir + "/output"; private FileSystem clusterFS = serverFS.get(0); @@ -96,7 +95,7 @@ public class ProcessUITest extends BaseUITestClass { bundles[0].setProcessPeriodicity(1, Frequency.TimeUnit.minutes); bundles[0].setProcessConcurrency(5); bundles[0].setInputFeedPeriodicity(1, Frequency.TimeUnit.minutes); - bundles[0].setInputFeedDataPath(feedInputPath + datePattern); + bundles[0].setInputFeedDataPath(feedInputPath + MINUTE_DATE_PATTERN); Process process = bundles[0].getProcessObject(); Inputs inputs = new Inputs(); Input input = new Input(); @@ -138,7 +137,7 @@ public class ProcessUITest extends BaseUITestClass { inputFeeds = LineageApiTest.generateFeeds(numInputFeeds, inputMerlin, Generator.getNameGenerator("infeed", inputMerlin.getName()), - Generator.getHadoopPathGenerator(feedInputPath, datePattern)); + Generator.getHadoopPathGenerator(feedInputPath, MINUTE_DATE_PATTERN)); int j = 0; for (FeedMerlin feed : inputFeeds) { bundles[0].addInputFeedToBundle("inputFeed" + j, feed.toString(), j++); @@ -146,7 +145,7 @@ public class ProcessUITest extends BaseUITestClass { outputFeeds = LineageApiTest.generateFeeds(numOutputFeeds, outputMerlin, Generator.getNameGenerator("outfeed", outputMerlin.getName()), - Generator.getHadoopPathGenerator(feedOutputPath, datePattern)); + Generator.getHadoopPathGenerator(feedOutputPath, MINUTE_DATE_PATTERN)); j = 0; for (FeedMerlin feed : outputFeeds) { bundles[0].addOutputFeedToBundle("outputFeed" + j, feed.toString(), j++);