falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Venkatesh Seetharam (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FALCON-50) Falcon replication having issue to s3
Date Thu, 14 Nov 2013 19:29:21 GMT

    [ https://issues.apache.org/jira/browse/FALCON-50?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13822799#comment-13822799
] 

Venkatesh Seetharam commented on FALCON-50:
-------------------------------------------

bq. Does the falcon in 0.4 version, now support the same cluster name to be defined in both
source and target definition of the feed xml?
Nope, the same validations exist in 0.4. But, this feature may not work in 0.4 and requires
you to have s3 URI in the write interface. 

> Falcon replication having issue to s3
> -------------------------------------
>
>                 Key: FALCON-50
>                 URL: https://issues.apache.org/jira/browse/FALCON-50
>             Project: Falcon
>          Issue Type: Bug
>          Components: general
>            Reporter: Biswajit Nayak
>              Labels: operability
>             Fix For: 0.4
>
>
> Overview :- Replication of data from one cluster to s3 is not working.
> Detail:-
> Replication feed was submitted and scheduled through falcon. While only the retention
bundle and co-ordinator (in oozie) are kicked up, could not see any sign of replication bundle
or co-ordinator in oozie logs. Dont see any error in logs.
> Below is the feed definition :-
> {noformat}
> <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
> <feed name="process-name" description="process-name" xmlns="uri:falcon:feed:0.1">
>     <frequency>hours(1)</frequency>
>     <late-arrival cut-off="hours(6)" />
>     <clusters>
>         <cluster name="cluster1" type="source">
>             <validity start="2013-07-17T00:00Z" end="2034-12-20T08:00Z"/>
>             <retention limit="days(24)" action="delete"/>
>         </cluster>
>         <cluster name="cluster2" type="target">
>             <validity start="2013-07-17T00:00Z" end="2034-12-20T08:00Z"/>
>             <retention limit="days(90)" action="delete"/>
>             <locations>
>                 <location type="data" path="s3://<bucket-name>/<path-folder>/${YEAR}-${MONTH}-${DAY}-${HOUR}/"/>
>             </locations>
>         </cluster>
>     </clusters>
>     <locations>
>         <location type="data" path="/path/to/copy/${YEAR}/${MONTH}/${DAY}/${HOUR}/"/>
>     </locations>
>     <ACL owner="user" group="group" permission="0x755"/>
>     <schema location="/schema/s1" provider="protobuf"/>
>     <properties>
>         <property name="queueName" value="queueName"/>
>         <property name="jobPriority" value="NORMAL"/>
>     </properties>
> </feed>
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message