falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pragya Mittal (JIRA)" <j...@apache.org>
Subject [jira] [Created] (FALCON-1526) Update happens twice in case of feed containing lifecycle
Date Fri, 09 Oct 2015 14:31:27 GMT
Pragya Mittal created FALCON-1526:
-------------------------------------

             Summary: Update happens twice in case of feed containing lifecycle
                 Key: FALCON-1526
                 URL: https://issues.apache.org/jira/browse/FALCON-1526
             Project: Falcon
          Issue Type: Bug
          Components: prism
    Affects Versions: 0.8
            Reporter: Pragya Mittal


If a feed containing lifecycle tag is updated then two bundles are created corresponding to
retention(source, target) and replication(target).

Feed definition is :
{code}
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<feed name="retention-feed1" description="Input File" xmlns="uri:falcon:feed:0.1">
    <partitions>
        <partition name="colo"/>
        <partition name="eventTime"/>
        <partition name="impressionHour"/>
        <partition name="pricingModel"/>
    </partitions>
    <frequency>minutes(5)</frequency>
    <timezone>UTC</timezone>
    <late-arrival cut-off="minutes(2)"/>
    <clusters>
        <cluster name="A969761bb-af20fc5f" type="source">
            <validity start="2015-10-09T14:00Z" end="2015-10-09T18:00Z"/>
            <retention limit="days(1000000)" action="delete"/>
        </cluster>
        <cluster name="A969761bb-69e50f1d" type="target">
            <validity start="2015-10-09T14:00Z" end="2015-10-09T15:00Z"/>
            <retention limit="days(1000000)" action="delete"/>
            <locations>
                <location type="data" path="/tmp/falcon-regression/LifecycleRetentionAndReplicationTest/target/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}"/>
            </locations>
        </cluster>
    </clusters>
    <locations>
        <location type="data" path="/tmp/falcon-regression/LifecycleRetentionAndReplicationTest/source/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}"/>
        <location type="stats" path="/data/regression/fetlrc/billing/stats"/>
        <location type="meta" path="/data/regression/fetlrc/billing/metadata"/>
    </locations>
    <ACL owner="pragya" group="dataqa" permission="*"/>
    <schema location="/databus/streams_local/click_rr/schema/" provider="protobuf"/>
    <properties>
        <property name="field1" value="value1"/>
        <property name="field2" value="value2"/>
    </properties>
    <lifecycle>
        <retention-stage>
            <frequency>minutes(10)</frequency>
            <queue>default</queue>
            <priority>NORMAL</priority>
            <properties>
                <property name="retention.policy.agebaseddelete.limit" value="minutes(15)"/>
            </properties>
        </retention-stage>
    </lifecycle>
</feed>


{code}


Updated log is :
{code}
dataqa@8RPCG32:/mnt/users/pragya/lifecycle$ fef -name retention-feed1 -update -file feed.xml
falcon/update/ua1/Updated successfully(feed) retention-feed1/Effective Time: 2015-10-09T14:25Z.
Old bundle id: 0000053-151009121634425-oozie-oozi-B. Old coordinator id: 0000054-151009121634425-oozie-oozi-C.
New bundle id: 0000064-151009121634425-oozie-oozi-B
(feed) retention-feed1/Effective Time: 2015-10-09T14:25Z. Old bundle id: 0000039-151009142953874-oozie-oozi-B.
Old coordinator id: 0000041-151009142953874-oozie-oozi-C,0000040-151009142953874-oozie-oozi-C.
New bundle id: 0000053-151009142953874-oozie-oozi-B

ua2/Updated successfully(feed) retention-feed1/Effective Time: 2015-10-09T14:25Z. Old bundle
id: 0000053-151009142953874-oozie-oozi-B. Old coordinator id: 0000054-151009142953874-oozie-oozi-C,0000055-151009142953874-oozie-oozi-C.
New bundle id: 0000059-151009142953874-oozie-oozi-B
(feed) retention-feed1/Effective Time: 2015-10-09T14:25Z. Old bundle id: 0000064-151009121634425-oozie-oozi-B.
Old coordinator id: 0000065-151009121634425-oozie-oozi-C. New coordinator id: 0000069-151009121634425-oozie-oozi-C


{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message