Return-Path: X-Original-To: apmail-falcon-dev-archive@minotaur.apache.org Delivered-To: apmail-falcon-dev-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 3D55E11459 for ; Thu, 31 Jul 2014 10:19:50 +0000 (UTC) Received: (qmail 8219 invoked by uid 500); 31 Jul 2014 10:19:49 -0000 Delivered-To: apmail-falcon-dev-archive@falcon.apache.org Received: (qmail 8177 invoked by uid 500); 31 Jul 2014 10:19:49 -0000 Mailing-List: contact dev-help@falcon.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@falcon.incubator.apache.org Delivered-To: mailing list dev@falcon.incubator.apache.org Received: (qmail 8166 invoked by uid 99); 31 Jul 2014 10:19:49 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 31 Jul 2014 10:19:49 +0000 X-ASF-Spam-Status: No, hits=-1998.4 required=5.0 tests=ALL_TRUSTED,HTML_MESSAGE,RP_MATCHES_RCVD X-Spam-Check-By: apache.org Received: from [140.211.11.3] (HELO mail.apache.org) (140.211.11.3) by apache.org (qpsmtpd/0.29) with SMTP; Thu, 31 Jul 2014 10:19:50 +0000 Received: (qmail 8008 invoked by uid 99); 31 Jul 2014 10:19:24 -0000 Received: from reviews-vm.apache.org (HELO reviews.apache.org) (140.211.11.40) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 31 Jul 2014 10:19:24 +0000 Received: from reviews.apache.org (localhost [127.0.0.1]) by reviews.apache.org (Postfix) with ESMTP id D4CD61D98F5; Thu, 31 Jul 2014 10:19:10 +0000 (UTC) Content-Type: multipart/alternative; boundary="===============1698010395362387326==" MIME-Version: 1.0 Subject: Re: Review Request 23844: FALCON-481 Simplify process parent workflow From: shwetha.gs@inmobi.com To: shwetha.gs@inmobi.com, "Falcon" , "Srikanth Sundarrajan" Date: Thu, 31 Jul 2014 10:19:10 -0000 Message-ID: <20140731101910.18793.70726@reviews.apache.org> X-ReviewBoard-URL: https://reviews.apache.org Auto-Submitted: auto-generated Sender: noreply@reviews.apache.org X-ReviewGroup: Falcon X-ReviewRequest-URL: https://reviews.apache.org/r/23844/ X-Sender: noreply@reviews.apache.org References: <20140730111105.21934.46530@reviews.apache.org> In-Reply-To: <20140730111105.21934.46530@reviews.apache.org> Reply-To: shwetha.gs@inmobi.com X-ReviewRequest-Repository: falcon-git X-Virus-Checked: Checked by ClamAV on apache.org --===============1698010395362387326== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23844/ ----------------------------------------------------------- (Updated July 31, 2014, 10:19 a.m.) Review request for Falcon. Changes ------- review comments incorporated Bugs: FALCON-481 https://issues.apache.org/jira/browse/FALCON-481 Repository: falcon-git Description ------- Currently, process parent workflow contains 2 decision nodes which take up oozie processing and oozie db. The refactoring creates customised workflows for fs/hcat, recordsize/without recordsize, pig/oozie/hive workflow engine processes. It also defines pre-processing and post-processing actions only once so that the definitions are consistent Diffs (updated) ----- common/src/main/java/org/apache/falcon/entity/EntityUtil.java 6f50829 oozie/src/main/java/org/apache/falcon/oozie/OozieBundleBuilder.java 62d95fa oozie/src/main/java/org/apache/falcon/oozie/OozieCoordinatorBuilder.java 5f483f0 oozie/src/main/java/org/apache/falcon/oozie/OozieEntityBuilder.java 1238b82 oozie/src/main/java/org/apache/falcon/oozie/OozieOrchestrationWorkflowBuilder.java 2ef162b oozie/src/main/java/org/apache/falcon/oozie/feed/FSReplicationWorkflowBuilder.java PRE-CREATION oozie/src/main/java/org/apache/falcon/oozie/feed/FeedReplicationCoordinatorBuilder.java 0b582ef oozie/src/main/java/org/apache/falcon/oozie/feed/FeedReplicationWorkflowBuilder.java 2537725 oozie/src/main/java/org/apache/falcon/oozie/feed/FeedRetentionCoordinatorBuilder.java ac38532 oozie/src/main/java/org/apache/falcon/oozie/feed/FeedRetentionWorkflowBuilder.java eee4fe9 oozie/src/main/java/org/apache/falcon/oozie/feed/HCatReplicationWorkflowBuilder.java PRE-CREATION oozie/src/main/java/org/apache/falcon/oozie/process/HiveProcessWorkflowBuilder.java 79a1883 oozie/src/main/java/org/apache/falcon/oozie/process/OozieProcessWorkflowBuilder.java 977d8c1 oozie/src/main/java/org/apache/falcon/oozie/process/PigProcessWorkflowBuilder.java 29f601d oozie/src/main/java/org/apache/falcon/oozie/process/ProcessExecutionCoordinatorBuilder.java e46ae6e oozie/src/main/java/org/apache/falcon/oozie/process/ProcessExecutionWorkflowBuilder.java 2e3a5c1 oozie/src/main/java/org/apache/falcon/util/OozieUtils.java 9e1c82d oozie/src/main/resources/action/feed/eviction-action.xml PRE-CREATION oozie/src/main/resources/action/feed/falcon-table-export.hql PRE-CREATION oozie/src/main/resources/action/feed/falcon-table-import.hql PRE-CREATION oozie/src/main/resources/action/feed/replication-action.xml PRE-CREATION oozie/src/main/resources/action/feed/table-cleanup.xml PRE-CREATION oozie/src/main/resources/action/feed/table-export.xml PRE-CREATION oozie/src/main/resources/action/feed/table-import.xml PRE-CREATION oozie/src/main/resources/action/post-process.xml PRE-CREATION oozie/src/main/resources/action/pre-process.xml PRE-CREATION oozie/src/main/resources/action/process/hive-action.xml PRE-CREATION oozie/src/main/resources/action/process/oozie-action.xml PRE-CREATION oozie/src/main/resources/action/process/pig-action.xml PRE-CREATION oozie/src/main/resources/workflow/falcon-table-export.hql 37fd1b7 oozie/src/main/resources/workflow/falcon-table-import.hql 653d580 oozie/src/main/resources/workflow/process-parent-workflow.xml 4a2495c oozie/src/main/resources/workflow/replication-workflow.xml 0748acf oozie/src/main/resources/workflow/retention-workflow.xml 5138865 oozie/src/test/java/org/apache/falcon/oozie/feed/OozieFeedWorkflowBuilderTest.java c99c36c oozie/src/test/java/org/apache/falcon/oozie/process/AbstractTestBase.java cc0c419 oozie/src/test/java/org/apache/falcon/oozie/process/OozieProcessWorkflowBuilderTest.java 3655af9 src/main/examples/entity/hcat/hcat-out-feed.xml 26d1f18 src/main/examples/entity/hcat/hcat-pig-process.xml 6ee17eb Diff: https://reviews.apache.org/r/23844/diff/ Testing ------- UTs + end to end tests for fs usecases Pending: hcat testing Thanks, shwethags --===============1698010395362387326==--