falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shwetha G S (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (FALCON-50) Falcon replication having issue to s3
Date Wed, 18 Sep 2013 06:14:55 GMT

     [ https://issues.apache.org/jira/browse/FALCON-50?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Shwetha G S updated FALCON-50:
------------------------------

    Description: 
Overview :- Replication of data from one cluster to s3 is not working.

Detail:-
Replication feed was submitted and scheduled through falcon. While only the retention bundle
and co-ordinator (in oozie) are kicked up, could not see any sign of replication bundle or
co-ordinator in oozie logs. Dont see any error in logs.

Below is the feed definition :-
{noformat}
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<feed name="process-name" description="process-name" xmlns="uri:falcon:feed:0.1">
    <frequency>hours(1)</frequency>
    <late-arrival cut-off="hours(6)" />
    <clusters>
        <cluster name="cluster1" type="source">
            <validity start="2013-07-17T00:00Z" end="2034-12-20T08:00Z"/>
            <retention limit="days(24)" action="delete"/>
        </cluster>
        <cluster name="cluster2" type="target">
            <validity start="2013-07-17T00:00Z" end="2034-12-20T08:00Z"/>
            <retention limit="days(90)" action="delete"/>
            <locations>
                <location type="data" path="s3://<bucket-name>/<path-folder>/${YEAR}-${MONTH}-${DAY}-${HOUR}/"/>
            </locations>
        </cluster>
    </clusters>
    <locations>
        <location type="data" path="/path/to/copy/${YEAR}/${MONTH}/${DAY}/${HOUR}/"/>
    </locations>
    <ACL owner="user" group="group" permission="0x755"/>
    <schema location="/schema/s1" provider="protobuf"/>
    <properties>
        <property name="queueName" value="queueName"/>
        <property name="jobPriority" value="NORMAL"/>
    </properties>
</feed>
{noformat}


  was:
Overview :- Replication of data from one cluster to s3 is not working.

Detail:-
Replication feed was submitted and scheduled through falcon. While only the retention bundle
and co-ordinator (in oozie) are kicked up, could not see any sign of replication bundle or
co-ordinator in oozie logs. Dont see any error in logs.

Below is the feed definition :-

?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<feed name="process-name" description="process-name" xmlns="uri:falcon:feed:0.1">
    <frequency>hours(1)</frequency>
    <late-arrival cut-off="hours(6)" />
    <clusters>
        <cluster name="cluster1" type="source">
            <validity start="2013-07-17T00:00Z" end="2034-12-20T08:00Z"/>
            <retention limit="days(24)" action="delete"/>
        </cluster>
        <cluster name="cluster2" type="target">
            <validity start="2013-07-17T00:00Z" end="2034-12-20T08:00Z"/>
            <retention limit="days(90)" action="delete"/>
            <locations>
                <location type="data" path="s3://<bucket-name>/<path-folder>/${YEAR}-${MONTH}-${DAY}-${HOUR}/"/>
            </locations>
        </cluster>
    </clusters>
    <locations>
        <location type="data" path="/path/to/copy/${YEAR}/${MONTH}/${DAY}/${HOUR}/"/>
    </locations>
    <ACL owner="user" group="group" permission="0x755"/>
    <schema location="/schema/s1" provider="protobuf"/>
    <properties>
        <property name="queueName" value="queueName"/>
        <property name="jobPriority" value="NORMAL"/>
    </properties>
</feed>
 

    
> Falcon replication having issue to s3
> -------------------------------------
>
>                 Key: FALCON-50
>                 URL: https://issues.apache.org/jira/browse/FALCON-50
>             Project: Falcon
>          Issue Type: Bug
>          Components: general
>            Reporter: Biswajit Nayak
>              Labels: operability
>
> Overview :- Replication of data from one cluster to s3 is not working.
> Detail:-
> Replication feed was submitted and scheduled through falcon. While only the retention
bundle and co-ordinator (in oozie) are kicked up, could not see any sign of replication bundle
or co-ordinator in oozie logs. Dont see any error in logs.
> Below is the feed definition :-
> {noformat}
> <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
> <feed name="process-name" description="process-name" xmlns="uri:falcon:feed:0.1">
>     <frequency>hours(1)</frequency>
>     <late-arrival cut-off="hours(6)" />
>     <clusters>
>         <cluster name="cluster1" type="source">
>             <validity start="2013-07-17T00:00Z" end="2034-12-20T08:00Z"/>
>             <retention limit="days(24)" action="delete"/>
>         </cluster>
>         <cluster name="cluster2" type="target">
>             <validity start="2013-07-17T00:00Z" end="2034-12-20T08:00Z"/>
>             <retention limit="days(90)" action="delete"/>
>             <locations>
>                 <location type="data" path="s3://<bucket-name>/<path-folder>/${YEAR}-${MONTH}-${DAY}-${HOUR}/"/>
>             </locations>
>         </cluster>
>     </clusters>
>     <locations>
>         <location type="data" path="/path/to/copy/${YEAR}/${MONTH}/${DAY}/${HOUR}/"/>
>     </locations>
>     <ACL owner="user" group="group" permission="0x755"/>
>     <schema location="/schema/s1" provider="protobuf"/>
>     <properties>
>         <property name="queueName" value="queueName"/>
>         <property name="jobPriority" value="NORMAL"/>
>     </properties>
> </feed>
> {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message