falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ajay Yadava (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (FALCON-2030) Enforce time partition pattern in the data location path in feed definition
Date Wed, 22 Jun 2016 12:34:57 GMT

    [ https://issues.apache.org/jira/browse/FALCON-2030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15344233#comment-15344233
] 

Ajay Yadava edited comment on FALCON-2030 at 6/22/16 12:34 PM:
---------------------------------------------------------------

[~bvellanki], [~me.venkatr]
I think I understood where the disconnect is. Allow me to explain. Each retention policy has
it's own validation, if some retention policy needs feed to be in time partition pattern then
we should do validation and throw error. 
However, this shouldn't be a blanket check and if the user is not using some feature which
makes sense only for time based partition, this should be allowed. We are planning to have
more such retention policies where feeds don't need to be in time partition style, in next
release of Falcon and then users won't need to write custom retention lifecycles.

Same holds for import/export case as well. If the feed without time partition doesn't make
sense for them, import / export should have such checks but if someone is not using import/export
those validations shouldn't kick in.

Makes sense?






was (Author: ajayyadava):
[~bvellanki], [~me.venkatr]
I think I understood where the disconnect is. Allow me to explain. Each retention policy has
it's own validation, if some retention policy needs feed to be in time partition pattern then
we should do validation and throw error. 
However, this shouldn't be a blanket check and if the user is not using some feature which
makes sense only for time based partition, this should be allowed. We are planning to have
more such retention policies where feeds don't need to be in time partition style, in next
release of Falcon and then users won't need to write custom retention lifecycles.

Same holds for import/export case as well. If it doesn't make sense for them, import / export
should have such checks but if someone is not using import/export those validations shouldn't
kick in.

Makes sense?





> Enforce time partition pattern in the data location path in feed definition 
> ----------------------------------------------------------------------------
>
>                 Key: FALCON-2030
>                 URL: https://issues.apache.org/jira/browse/FALCON-2030
>             Project: Falcon
>          Issue Type: Improvement
>          Components: feed
>            Reporter: Venkatesan Ramachandran
>            Assignee: Venkatesan Ramachandran
>
> In feed definition, data location can be specified without time series pattern like below:
>    <locations>
>         <location type="data" path="/tmp/falcon-regression/RetentionTest/testFolders/"/>
>         <location type="stats" path="/projects/falcon/clicksStats"/>
>         <location type="meta" path="/projects/falcon/clicksMetaData"/>
>     </locations>



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message