falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pallavi Rao (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FALCON-2030) Enforce time partition pattern in the data location path in feed definition
Date Mon, 20 Jun 2016 04:45:05 GMT

    [ https://issues.apache.org/jira/browse/FALCON-2030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15338997#comment-15338997

Pallavi Rao commented on FALCON-2030:

[~me.venkatr], I did a quick scan of the Inmobi feeds and found that there are indeed a few
adhoc feeds 6 out of ~1000 that were created without any time pattern for some adhoc requirements
for replication and/or retention. These jobs could have been created with a time pattern,
but, users created without one as it was simpler. 

Not a big case against validation. However, if we put in this validation, it will be backward
incompatible. What if user wants to update the exiting feeds (that don't follow the time pattern),
do we fail then?

> Enforce time partition pattern in the data location path in feed definition 
> ----------------------------------------------------------------------------
>                 Key: FALCON-2030
>                 URL: https://issues.apache.org/jira/browse/FALCON-2030
>             Project: Falcon
>          Issue Type: Improvement
>          Components: feed
>            Reporter: Venkatesan Ramachandran
>            Assignee: Venkatesan Ramachandran
> In feed definition, data location can be specified without time series pattern like below:
>    <locations>
>         <location type="data" path="/tmp/falcon-regression/RetentionTest/testFolders/"/>
>         <location type="stats" path="/projects/falcon/clicksStats"/>
>         <location type="meta" path="/projects/falcon/clicksMetaData"/>
>     </locations>

This message was sent by Atlassian JIRA

View raw message