falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Venkat Ramachandran (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (FALCON-36) Ability to ingest data from databases
Date Tue, 04 Aug 2015 05:45:05 GMT

    [ https://issues.apache.org/jira/browse/FALCON-36?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14653099#comment-14653099
] 

Venkat Ramachandran edited comment on FALCON-36 at 8/4/15 5:44 AM:
-------------------------------------------------------------------

[~ajayyadava] Exposing specific entities like DATABASE enables an user/admin to ask Falcon
to list all databases being used in production for audit purposes. Its not possible with a
DATASOURCE entity unless Falcon entity supports some kind of inheritance. Also, from usability
stand point, having specific entity type make it clear and concrete and avoid confusions.

There is no right way to explode DATASOURCE, but the current thinking is to, based on customer
use cases, classify into DATABASES, FILESYSTEMS, STREAMS. 




was (Author: me.venkatr):
[~ajayyadava] Exposing specific entities like DATABASE enables an user/admin to ask Falcon
to list all databases being used in production for audit purposes. Its not possible with a
DATASOURCE entity unless Falcon entity supports some kind of inheritance. Also, from usability
stand point, having specific entity type make it clear and concrete and avoid confusions.



> Ability to ingest data from databases
> -------------------------------------
>
>                 Key: FALCON-36
>                 URL: https://issues.apache.org/jira/browse/FALCON-36
>             Project: Falcon
>          Issue Type: Improvement
>          Components: acquisition
>    Affects Versions: 0.3
>            Reporter: Venkatesh Seetharam
>            Assignee: Venkat Ramachandran
>         Attachments: FALCON-36.patch, FALCON-36.patch.2, FALCON-36.rebase.patch, FALCON-36.review.patch,
Falcon Data Ingestion - Proposal.docx, falcon-36.xsd.patch.1
>
>
> Attempt to address data import from RDBMS into hadoop and export of data from Hadoop
into RDBMS. The plan is to use sqoop 1.x to materialize data motion from/to RDBMS to/from
HDFS. Hive will not be integrated in the first pass until Falcon has a first class integration
with HCatalog.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message