falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shaik Idris Ali (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FALCON-287) Record lineage information in post processing
Date Wed, 05 Mar 2014 15:17:43 GMT

    [ https://issues.apache.org/jira/browse/FALCON-287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13920953#comment-13920953
] 

Shaik Idris Ali commented on FALCON-287:
----------------------------------------

May be I missed the actual use-case of lineage, why do we need to persist it and who are the
consumers of this information.

Secondly, instead of FalconPostProcessing storing this data on HDFS, which might further slowdown
user workflow.
A service can subscribe to FALCON.ENTITY.TOPIC and record the details in HDFS or anywhere.
I think we discussed about this doing in ProcessSubscriberService.

Can we discuss this over a chat? 11 PM IST.


> Record lineage information in post processing
> ---------------------------------------------
>
>                 Key: FALCON-287
>                 URL: https://issues.apache.org/jira/browse/FALCON-287
>             Project: Falcon
>          Issue Type: Sub-task
>    Affects Versions: 0.5
>            Reporter: Venkatesh Seetharam
>            Assignee: Venkatesh Seetharam
>              Labels: lineage
>         Attachments: FALCON-287-v1.patch, FALCON-287-v2.patch, FALCON-287.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message