falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ajay Yadava (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FALCON-1526) Entity Updates are happening multiple times in a single update command
Date Wed, 14 Oct 2015 07:20:06 GMT

    [ https://issues.apache.org/jira/browse/FALCON-1526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14956414#comment-14956414
] 

Ajay Yadava commented on FALCON-1526:
-------------------------------------

[~pragya.mittal] Thanks for the detailed report. You are right that this bug is not specific
to lifecycle, it is specific to distributed mode and is general to all feed/process updates.

Can you please try it with deploy.properties containing the following two properties?
*.deploy.mode=distributed
*.domain=falcon



> Entity Updates are happening multiple times in a single update command
> ----------------------------------------------------------------------
>
>                 Key: FALCON-1526
>                 URL: https://issues.apache.org/jira/browse/FALCON-1526
>             Project: Falcon
>          Issue Type: Bug
>          Components: prism
>    Affects Versions: 0.8
>            Reporter: Pragya Mittal
>            Assignee: Ajay Yadava
>
> If a feed containing lifecycle tag is updated then two bundles are created corresponding
to retention(source, target) and replication(target).
> Feed definition is :
> {code}
> <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
> <feed name="retention-feed1" description="Input File" xmlns="uri:falcon:feed:0.1">
>     <partitions>
>         <partition name="colo"/>
>         <partition name="eventTime"/>
>         <partition name="impressionHour"/>
>         <partition name="pricingModel"/>
>     </partitions>
>     <frequency>minutes(5)</frequency>
>     <timezone>UTC</timezone>
>     <late-arrival cut-off="minutes(2)"/>
>     <clusters>
>         <cluster name="A969761bb-af20fc5f" type="source">
>             <validity start="2015-10-09T14:00Z" end="2015-10-09T18:00Z"/>
>             <retention limit="days(1000000)" action="delete"/>
>         </cluster>
>         <cluster name="A969761bb-69e50f1d" type="target">
>             <validity start="2015-10-09T14:00Z" end="2015-10-09T15:00Z"/>
>             <retention limit="days(1000000)" action="delete"/>
>             <locations>
>                 <location type="data" path="/tmp/falcon-regression/LifecycleRetentionAndReplicationTest/target/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}"/>
>             </locations>
>         </cluster>
>     </clusters>
>     <locations>
>         <location type="data" path="/tmp/falcon-regression/LifecycleRetentionAndReplicationTest/source/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}"/>
>         <location type="stats" path="/data/regression/fetlrc/billing/stats"/>
>         <location type="meta" path="/data/regression/fetlrc/billing/metadata"/>
>     </locations>
>     <ACL owner="pragya" group="dataqa" permission="*"/>
>     <schema location="/databus/streams_local/click_rr/schema/" provider="protobuf"/>
>     <properties>
>         <property name="field1" value="value1"/>
>         <property name="field2" value="value2"/>
>     </properties>
>     <lifecycle>
>         <retention-stage>
>             <frequency>minutes(10)</frequency>
>             <queue>default</queue>
>             <priority>NORMAL</priority>
>             <properties>
>                 <property name="retention.policy.agebaseddelete.limit" value="minutes(15)"/>
>             </properties>
>         </retention-stage>
>     </lifecycle>
> </feed>
> {code}
> Update response log is :
> {noformat}
> dataqa@8RPCG32:/mnt/users/pragya/lifecycle$ fef -name retention-feed1 -update -file feed.xml
> falcon/update/ua1/Updated successfully(feed) retention-feed1/Effective Time: 2015-10-09T14:25Z.
Old bundle id: 0000053-151009121634425-oozie-oozi-B. Old coordinator id: 0000054-151009121634425-oozie-oozi-C.
New bundle id: 0000064-151009121634425-oozie-oozi-B
> (feed) retention-feed1/Effective Time: 2015-10-09T14:25Z. Old bundle id: 0000039-151009142953874-oozie-oozi-B.
Old coordinator id: 0000041-151009142953874-oozie-oozi-C,0000040-151009142953874-oozie-oozi-C.
New bundle id: 0000053-151009142953874-oozie-oozi-B
> ua2/Updated successfully(feed) retention-feed1/Effective Time: 2015-10-09T14:25Z. Old
bundle id: 0000053-151009142953874-oozie-oozi-B. Old coordinator id: 0000054-151009142953874-oozie-oozi-C,0000055-151009142953874-oozie-oozi-C.
New bundle id: 0000059-151009142953874-oozie-oozi-B
> (feed) retention-feed1/Effective Time: 2015-10-09T14:25Z. Old bundle id: 0000064-151009121634425-oozie-oozi-B.
Old coordinator id: 0000065-151009121634425-oozie-oozi-C. New coordinator id: 0000069-151009121634425-oozie-oozi-C
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message