falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Venkatesh Seetharam (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FALCON-287) Record lineage information in post processing
Date Wed, 05 Mar 2014 19:50:46 GMT

    [ https://issues.apache.org/jira/browse/FALCON-287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13921305#comment-13921305
] 

Venkatesh Seetharam commented on FALCON-287:
--------------------------------------------

bq. is the existing logDir used for storing lineage info?
Yes. The sequence is:
1. persist metadata into hdfs
2. notify falcon that wf completed successfully
3. post success message in falcon, construct graph for the metadata
4. when log clean up services comes up, it cleans up stale metadata on hdfs


> Record lineage information in post processing
> ---------------------------------------------
>
>                 Key: FALCON-287
>                 URL: https://issues.apache.org/jira/browse/FALCON-287
>             Project: Falcon
>          Issue Type: Sub-task
>    Affects Versions: 0.5
>            Reporter: Venkatesh Seetharam
>            Assignee: Venkatesh Seetharam
>              Labels: lineage
>         Attachments: FALCON-287-v1.patch, FALCON-287-v2.patch, FALCON-287.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message