falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Venkatesh Seetharam (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FALCON-288) Persist lineage information into a persistent store
Date Wed, 05 Feb 2014 07:36:09 GMT

    [ https://issues.apache.org/jira/browse/FALCON-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13891885#comment-13891885
] 

Venkatesh Seetharam commented on FALCON-288:
--------------------------------------------

We need to discuss 2 things:

* persistent store - rdbms vs a graph db
* object model based on the store we choose

I tried to model in both and the queries for getting a graph for a single execution with multiple
instances for an input involved several inner joins. However, its quite straightforward in
graphs.

Currently, I have each entity as a vertex with name and type as indexed properties. Each instance
for a given entity is also a vertex with edges to the entity and the instances it uses or
generates with name as the indexed property along with user and timestamp.

Thoughts?



> Persist lineage information into a persistent store
> ---------------------------------------------------
>
>                 Key: FALCON-288
>                 URL: https://issues.apache.org/jira/browse/FALCON-288
>             Project: Falcon
>          Issue Type: Sub-task
>    Affects Versions: 0.5
>            Reporter: Venkatesh Seetharam
>              Labels: lineage
>
> Need to evaluate the store - rdbms vs graph db. Leaning towards latter since the data
is hierarchical.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message