falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sowmya Ramesh" <sram...@hortonworks.com>
Subject Re: Review Request 38465: FALCON-1459 : Ability to import from database
Date Wed, 23 Sep 2015 03:26:00 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38465/#review100036
-----------------------------------------------------------


Not sure if the patch uploaded is correct. Can't find DatasourceEntityParser/DataSourceHelper
classes. Will continue review after latest correct patch is uploaded.


client/src/main/java/org/apache/falcon/metadata/RelationshipType.java (line 30)
<https://reviews.apache.org/r/38465/#comment157124>

    Should this be DATASOURCE_ENTITY?



client/src/main/resources/feed-0.1.xsd (line 476)
<https://reviews.apache.org/r/38465/#comment157256>

    Can't find the documentation for fields-type in xsd. Also can you move this before arguments
type so that all elements related to source are after one another?



common/src/main/java/org/apache/falcon/entity/EntityUtil.java (line 950)
<https://reviews.apache.org/r/38465/#comment157257>

    minor nit : I see that we already import org.apache.falcon.entity.v0.cluster. Can we just
use cluster here?



common/src/main/java/org/apache/falcon/entity/FeedHelper.java (line 760)
<https://reviews.apache.org/r/38465/#comment157258>

    Can't find DatasourceHelper file.



common/src/main/java/org/apache/falcon/entity/FeedHelper.java (line 770)
<https://reviews.apache.org/r/38465/#comment157259>

    Since these are related to data source it shoudl be moved to DatasourceHelper.



common/src/main/java/org/apache/falcon/entity/FeedHelper.java (line 785)
<https://reviews.apache.org/r/38465/#comment157267>

    Can getSource return null or does xsd validation fail if not passed? Same for getTableName
and getExtract and others



common/src/main/java/org/apache/falcon/entity/FeedHelper.java (line 815)
<https://reviews.apache.org/r/38465/#comment157265>

    Can getsource, getextract return null?



common/src/main/java/org/apache/falcon/entity/FeedHelper.java (line 829)
<https://reviews.apache.org/r/38465/#comment157266>

    same as above



common/src/main/java/org/apache/falcon/entity/FeedHelper.java (line 845)
<https://reviews.apache.org/r/38465/#comment157268>

    Does the time zone matter here?



common/src/main/java/org/apache/falcon/entity/FeedHelper.java (line 873)
<https://reviews.apache.org/r/38465/#comment157270>

    In xsd minoccurs is 0 for arguments. Can getArguments return null and cause NPE?



common/src/main/java/org/apache/falcon/entity/parser/EntityParserFactory.java (line 49)
<https://reviews.apache.org/r/38465/#comment157274>

    can't find DatasourceEntityParser


- Sowmya Ramesh


On Sept. 19, 2015, 12:26 a.m., Venkatesan Ramachandran wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38465/
> -----------------------------------------------------------
> 
> (Updated Sept. 19, 2015, 12:26 a.m.)
> 
> 
> Review request for Falcon, Ajay Yadava, Balu Vellanki, Peeyush Bishnoi, Sowmya Ramesh,
and Venkat Ranganathan.
> 
> 
> Repository: falcon-git
> 
> 
> Description
> -------
> 
> FALCON-1459 : Ability to import from database
> 
> 
> Diffs
> -----
> 
>   client/src/main/java/org/apache/falcon/LifeCycle.java 58a2a6c 
>   client/src/main/java/org/apache/falcon/Tag.java beeb812 
>   client/src/main/java/org/apache/falcon/entity/v0/EntityType.java 0657124 
>   client/src/main/java/org/apache/falcon/metadata/RelationshipType.java f034772 
>   client/src/main/resources/feed-0.1.xsd 4ff8baa 
>   client/src/main/resources/jaxb-binding.xjb f644f40 
>   common/src/main/java/org/apache/falcon/entity/EntityUtil.java 2f05b1f 
>   common/src/main/java/org/apache/falcon/entity/FeedHelper.java 572923b 
>   common/src/main/java/org/apache/falcon/entity/parser/EntityParserFactory.java 5a33201

>   common/src/main/java/org/apache/falcon/entity/parser/FeedEntityParser.java 992fc51

>   common/src/main/java/org/apache/falcon/entity/store/ConfigurationStore.java e27187b

>   common/src/main/java/org/apache/falcon/entity/v0/EntityGraph.java bd4c6cf 
>   common/src/main/java/org/apache/falcon/entity/v0/EntityIntegrityChecker.java bd32852

>   common/src/main/java/org/apache/falcon/metadata/EntityRelationshipGraphBuilder.java
8c3876c 
>   common/src/main/java/org/apache/falcon/workflow/WorkflowExecutionContext.java 4454239

>   common/src/test/java/org/apache/falcon/entity/AbstractTestBase.java 6179855 
>   common/src/test/java/org/apache/falcon/entity/EntityTypeTest.java 640e87d 
>   common/src/test/java/org/apache/falcon/entity/FeedHelperTest.java c70cfcc 
>   common/src/test/java/org/apache/falcon/entity/parser/FeedEntityParserTest.java d203b7c

>   common/src/test/java/org/apache/falcon/entity/v0/EntityGraphTest.java 3863b11 
>   falcon-regression/merlin-core/src/main/java/org/apache/falcon/regression/core/util/HiveAssert.java
2a934b5 
>   oozie/src/main/java/org/apache/falcon/oozie/OozieCoordinatorBuilder.java 85f5330 
>   oozie/src/main/java/org/apache/falcon/oozie/OozieOrchestrationWorkflowBuilder.java
3213a70 
>   oozie/src/main/java/org/apache/falcon/oozie/feed/FeedBundleBuilder.java b819dee 
>   pom.xml 646de69 
>   webapp/pom.xml 828f7f5 
>   webapp/src/test/java/org/apache/falcon/resource/TestContext.java f031137 
> 
> Diff: https://reviews.apache.org/r/38465/diff/
> 
> 
> Testing
> -------
> 
> * Unit tests
> * Integration tests
> * Manual tests
>   * Setup MySQL, create table and populate
>   * Create datasource and feed entity with import policy in Falcon  
>   * Made sure the data lands up in the HDFS.
> 
> 
> Thanks,
> 
> Venkatesan Ramachandran
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message