Return-Path: X-Original-To: apmail-falcon-dev-archive@minotaur.apache.org Delivered-To: apmail-falcon-dev-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 64509185B9 for ; Fri, 23 Oct 2015 21:03:05 +0000 (UTC) Received: (qmail 30455 invoked by uid 500); 23 Oct 2015 21:03:05 -0000 Delivered-To: apmail-falcon-dev-archive@falcon.apache.org Received: (qmail 30412 invoked by uid 500); 23 Oct 2015 21:03:05 -0000 Mailing-List: contact dev-help@falcon.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@falcon.apache.org Delivered-To: mailing list dev@falcon.apache.org Received: (qmail 30397 invoked by uid 99); 23 Oct 2015 21:03:05 -0000 Received: from Unknown (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 23 Oct 2015 21:03:05 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 97ACAC5176 for ; Fri, 23 Oct 2015 21:03:04 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 3.982 X-Spam-Level: *** X-Spam-Status: No, score=3.982 tagged_above=-999 required=6.31 tests=[HEADER_FROM_DIFFERENT_DOMAINS=0.001, HTML_MESSAGE=3, KAM_LAZY_DOMAIN_SECURITY=1, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=0.001] autolearn=disabled Received: from mx1-us-east.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id meA-worXBV9u for ; Fri, 23 Oct 2015 21:03:03 +0000 (UTC) Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx1-us-east.apache.org (ASF Mail Server at mx1-us-east.apache.org) with SMTP id F098842BEB for ; Fri, 23 Oct 2015 21:03:02 +0000 (UTC) Received: (qmail 30381 invoked by uid 99); 23 Oct 2015 21:03:02 -0000 Received: from reviews-vm.apache.org (HELO reviews.apache.org) (140.211.11.40) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 23 Oct 2015 21:03:02 +0000 Received: from reviews.apache.org (localhost [127.0.0.1]) by reviews.apache.org (Postfix) with ESMTP id 5DE78279BF9; Fri, 23 Oct 2015 21:03:02 +0000 (UTC) Content-Type: multipart/alternative; boundary="===============6387202493579761914==" MIME-Version: 1.0 Subject: Re: Review Request 38465: FALCON-1459 : Ability to import from database From: "Balu Vellanki" To: "Venkat Ranganathan" , "Balu Vellanki" , "Peeyush Bishnoi" , "Sowmya Ramesh" , "Ajay Yadava" Cc: "Venkatesan Ramachandran" , "Falcon" Date: Fri, 23 Oct 2015 21:03:02 -0000 Message-ID: <20151023210302.1655.23930@reviews.apache.org> X-ReviewBoard-URL: https://reviews.apache.org/ Auto-Submitted: auto-generated Sender: "Balu Vellanki" X-ReviewGroup: Falcon X-Auto-Response-Suppress: DR, RN, OOF, AutoReply X-ReviewRequest-URL: https://reviews.apache.org/r/38465/ X-Sender: "Balu Vellanki" References: <20151023160141.1657.4154@reviews.apache.org> In-Reply-To: <20151023160141.1657.4154@reviews.apache.org> Reply-To: "Balu Vellanki" X-ReviewRequest-Repository: falcon-git --===============6387202493579761914== MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/38465/#review103864 ----------------------------------------------------------- Apart from comments made by Sowmya, rest of the patch looks good to me. - Balu Vellanki On Oct. 23, 2015, 4:01 p.m., Venkatesan Ramachandran wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/38465/ > ----------------------------------------------------------- > > (Updated Oct. 23, 2015, 4:01 p.m.) > > > Review request for Falcon, Ajay Yadava, Balu Vellanki, Peeyush Bishnoi, Sowmya Ramesh, and Venkat Ranganathan. > > > Repository: falcon-git > > > Description > ------- > > FALCON-1459 : Ability to import from database > > > Diffs > ----- > > client/src/main/java/org/apache/falcon/LifeCycle.java 58a2a6c > client/src/main/java/org/apache/falcon/Tag.java beeb812 > client/src/main/java/org/apache/falcon/entity/v0/EntityType.java 0657124 > client/src/main/java/org/apache/falcon/metadata/RelationshipType.java f034772 > client/src/main/resources/datasource-0.1.xsd PRE-CREATION > client/src/main/resources/feed-0.1.xsd 77b8f4b > client/src/main/resources/jaxb-binding.xjb 6f1d6c7 > client/src/main/resources/mysql_database.xml PRE-CREATION > common/src/main/java/org/apache/falcon/entity/DatasourceHelper.java PRE-CREATION > common/src/main/java/org/apache/falcon/entity/EntityUtil.java ceefb17 > common/src/main/java/org/apache/falcon/entity/FeedHelper.java 5c252a8 > common/src/main/java/org/apache/falcon/entity/parser/DatasourceEntityParser.java PRE-CREATION > common/src/main/java/org/apache/falcon/entity/parser/EntityParserFactory.java 5a33201 > common/src/main/java/org/apache/falcon/entity/parser/FeedEntityParser.java c5cfdd2 > common/src/main/java/org/apache/falcon/entity/store/ConfigurationStore.java e27187b > common/src/main/java/org/apache/falcon/entity/v0/EntityGraph.java bd4c6cf > common/src/main/java/org/apache/falcon/entity/v0/EntityIntegrityChecker.java bd32852 > common/src/main/java/org/apache/falcon/metadata/EntityRelationshipGraphBuilder.java 8c3876c > common/src/main/java/org/apache/falcon/metadata/InstanceRelationshipGraphBuilder.java f485764 > common/src/main/java/org/apache/falcon/metadata/MetadataMappingService.java 56fbde0 > common/src/main/java/org/apache/falcon/metadata/RelationshipLabel.java 5b312da > common/src/main/java/org/apache/falcon/util/HdfsClassLoader.java PRE-CREATION > common/src/main/java/org/apache/falcon/workflow/WorkflowExecutionArgs.java ac7140c > common/src/main/java/org/apache/falcon/workflow/WorkflowExecutionContext.java b870e3a > common/src/test/java/org/apache/falcon/entity/AbstractTestBase.java aab9cee > common/src/test/java/org/apache/falcon/entity/EntityTypeTest.java 640e87d > common/src/test/java/org/apache/falcon/entity/FeedHelperTest.java 4020d36 > common/src/test/java/org/apache/falcon/entity/parser/DatasourceEntityParserTest.java PRE-CREATION > common/src/test/java/org/apache/falcon/entity/parser/FeedEntityParserTest.java 905be68 > common/src/test/java/org/apache/falcon/entity/v0/EntityGraphTest.java 3863b11 > common/src/test/resources/config/datasource/datasource-0.1.xml PRE-CREATION > common/src/test/resources/config/datasource/datasource-file-0.1.xml PRE-CREATION > common/src/test/resources/config/datasource/datasource-invalid-0.1.xml PRE-CREATION > common/src/test/resources/config/feed/feed-import-0.1.xml PRE-CREATION > common/src/test/resources/config/feed/feed-import-exclude-fields-0.1.xml PRE-CREATION > common/src/test/resources/config/feed/feed-import-invalid-0.1.xml PRE-CREATION > common/src/test/resources/config/feed/feed-import-noargs-0.1.xml PRE-CREATION > docs/src/site/twiki/EntitySpecification.twiki e07fe12 > docs/src/site/twiki/FalconCLI.twiki d994866 > messaging/src/main/java/org/apache/falcon/messaging/JMSMessageProducer.java 515562a > oozie/src/main/java/org/apache/falcon/oozie/DatabaseImportWorkflowBuilder.java PRE-CREATION > oozie/src/main/java/org/apache/falcon/oozie/FeedImportCoordinatorBuilder.java PRE-CREATION > oozie/src/main/java/org/apache/falcon/oozie/ImportWorkflowBuilder.java PRE-CREATION > oozie/src/main/java/org/apache/falcon/oozie/OozieCoordinatorBuilder.java a04ae95 > oozie/src/main/java/org/apache/falcon/oozie/OozieOrchestrationWorkflowBuilder.java 3213a70 > oozie/src/main/java/org/apache/falcon/oozie/feed/FSReplicationWorkflowBuilder.java 0dc09ee > oozie/src/main/java/org/apache/falcon/oozie/feed/FeedBundleBuilder.java 9e55edf > oozie/src/main/java/org/apache/falcon/oozie/feed/FeedRetentionWorkflowBuilder.java f953557 > oozie/src/main/java/org/apache/falcon/oozie/feed/HCatReplicationWorkflowBuilder.java 6e2a631 > oozie/src/main/java/org/apache/falcon/oozie/process/ProcessExecutionWorkflowBuilder.java 3f49adb > oozie/src/main/resources/action/feed/import-sqoop-database-action.xml PRE-CREATION > oozie/src/main/resources/action/post-process.xml df0d286 > pom.xml 8cd3c3c > webapp/pom.xml 8891e5f > webapp/src/test/java/org/apache/falcon/lifecycle/FeedImportIT.java PRE-CREATION > webapp/src/test/java/org/apache/falcon/resource/TestContext.java d067dee > webapp/src/test/java/org/apache/falcon/util/HsqldbTestUtils.java PRE-CREATION > webapp/src/test/resources/datasource-template.xml PRE-CREATION > webapp/src/test/resources/feed-template3.xml PRE-CREATION > > Diff: https://reviews.apache.org/r/38465/diff/ > > > Testing > ------- > > * Unit tests > * Integration tests > * Manual tests > * Setup MySQL, create table and populate > * Create datasource and feed entity with import policy in Falcon > * Made sure the data lands up in the HDFS. > > > Thanks, > > Venkatesan Ramachandran > > --===============6387202493579761914==--