falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Venkatesh Seetharam (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FALCON-580) Mandate date pattern for the feed path in the xsd
Date Wed, 03 Sep 2014 17:58:52 GMT

    [ https://issues.apache.org/jira/browse/FALCON-580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14120170#comment-14120170
] 

Venkatesh Seetharam commented on FALCON-580:
--------------------------------------------

Makes sense to write up use cases and high level approach to how this is going to work. 

* How will Oozie data availability triggers work now?

bq. Implement age based eviction
This should now work on files instead of partitions. Becomes obtuse.

bq. In general work out better ways to identify change set for replication if feed is mutable
Can be absorbed in DistCp

bq. Ensure there are no gaps for HCat tables when this is implemented for path based feeds
This implies we'd have support for tables with out partitions. Imports and exports will always
dump full table. Imports might fail as well.

> Mandate date pattern for the feed path in the xsd
> -------------------------------------------------
>
>                 Key: FALCON-580
>                 URL: https://issues.apache.org/jira/browse/FALCON-580
>             Project: Falcon
>          Issue Type: Bug
>            Reporter: Sowmya Ramesh
>
>  The granularity of date pattern in the feed path should be atleast that of a frequency
of a feed. This should be mandated in the feed xsd.
> e.g.: 
> {noformat}
> Valid format: <location type="data" path="/hdfsDataLocation/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}"/>
> Invalid format: <location type="data" path="/hdfsDataLocation"/>
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message