falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Suhas Vasu (JIRA)" <j...@apache.org>
Subject [jira] [Closed] (FALCON-494) update feature broken
Date Tue, 08 Jul 2014 09:02:04 GMT

     [ https://issues.apache.org/jira/browse/FALCON-494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Suhas Vasu closed FALCON-494.
-----------------------------


> update feature broken 
> ----------------------
>
>                 Key: FALCON-494
>                 URL: https://issues.apache.org/jira/browse/FALCON-494
>             Project: Falcon
>          Issue Type: Bug
>          Components: general
>    Affects Versions: 0.6
>         Environment: QA
>            Reporter: Samarth Gupta
>            Assignee: Shwetha G S
>            Priority: Blocker
>             Fix For: 0.4, 0.5
>
>         Attachments: FALCON-494.patch
>
>
> when we update an process / feed , the update take place from current time, and all the
future instance of the old coordinator are moved to new coord with new definition. 
> However with the latest falcon, the coord which is getting created is having all instances
which have have been already processed by old coodinator. Resulting in rerunning on those
instances  which are already in RUNNING / READY / WAITING state in old coord. 
> in the logs, following lines can be seen which says the new start time :
> {code}
> 2014-06-25 06:36:40,819 INFO  - [1871593525@qtp-1222207508-0:dataqa:POST//sync/update/process/agregator-coord16-bf1e8076-16fe-4fba-bb76-b780ec3a05b0-e92d8b59-42ef-41ab-9625-596df66d2395
1118a58a-4a4e-4052-afea-40d4960470f3] ~ Submitted 0000328-140624044727425-oozie-oozi-B on
cluster corp-2babc92c-129f-422e-b836-17e286eeaf0111-9b497e34-e892-4e9b-a5ee-f9a417e55f1e with
properties: {nameNode=hdfs://192.168.138.182:8020, oozie.use.system.libpath=true, field2=value2,
jobTracker=192.168.138.182:8021, field1=value1, falcon.libpath=/projectsTest/ivory/working/lib,
colo.name=ua3, oozie.bundle.application.path=${nameNode}/projects/ivory/staging/falcon/workflows/process/agregator-coord16-bf1e8076-16fe-4fba-bb76-b780ec3a05b0-e92d8b59-42ef-41ab-9625-596df66d2395/1403678197513,
user.name=dataqa} (OozieWorkflowEngine:1316)
> 2014-06-25 06:36:40,827 INFO  - [1871593525@qtp-1222207508-0:dataqa:POST//sync/update/process/agregator-coord16-bf1e8076-16fe-4fba-bb76-b780ec3a05b0-e92d8b59-42ef-41ab-9625-596df66d2395
1118a58a-4a4e-4052-afea-40d4960470f3] ~ New bundle 0000328-140624044727425-oozie-oozi-B scheduled
successfully with start time 2014-06-25T06:39Z (OozieWorkflowEngine:1131)
> 2014-06-25 06:36:40,828 DEBUG - [1871593525@qtp-1222207508-0:dataqa:POST//sync/update/process/agregator-coord16-bf1e8076-16fe-4fba-bb76-b780ec3a05b0-e92d8b59-42ef-41ab-9625-596df66d2395
1118a58a-4a4e-4052-afea-40d4960470f3] ~ Updating endtime of coord 0000323-140624044727425-oozie-oozi-C
to 2014-06-25T06:39Z on cluster corp-2babc92c-129f-422e-b836-17e286eeaf0111-9b497e34-e892-4e9b-a5ee-f9a417e55f1e
(OozieWorkflowEngine:1055)
> {code}
> but in the coord xml the start time is till the original process time :
> {code}
> 2014-06-25 06:36:39,925 DEBUG - [1871593525@qtp-1222207508-0:dataqa:POST//sync/update/process/agregator-coord16-bf1e8076-16fe-4fba-bb76-b780ec3a05b0-e92d8b59-42ef-41ab-9625-596df66d2395
1118a58a-4a4e-4052-afea-40d4960470f3] ~ Writing definition to /projects/ivory/staging/falcon/workflows/process/agregator-coord16-bf1e8076-16fe-4fba-bb76-b780ec3a05b0-e92d8b59-42ef-41ab-9625-596df66d2395/1403678197513/DEFAULT/coordinator.xml
on cluster corp-2babc92c-129f-422e-b836-17e286eeaf0111-9b497e34-e892-4e9b-a5ee-f9a417e55f1e
(OozieWorkflowBuilder:338)
> 2014-06-25 06:36:39,925 DEBUG - [1871593525@qtp-1222207508-0:dataqa:POST//sync/update/process/agregator-coord16-bf1e8076-16fe-4fba-bb76-b780ec3a05b0-e92d8b59-42ef-41ab-9625-596df66d2395
1118a58a-4a4e-4052-afea-40d4960470f3] ~ <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
> <coordinator-app name="FALCON_PROCESS_DEFAULT_agregator-coord16-bf1e8076-16fe-4fba-bb76-b780ec3a05b0-e92d8b59-42ef-41ab-9625-596df66d2395"
frequency="${coord:minutes(1)}" start="2014-06-25T06:34Z" end="2014-06-25T06:44Z" timezone="UTC"
xmlns="uri:oozie:coordinator:0.3">
>     <controls>
>         <timeout>30</timeout>
>         <concurrency>1</concurrency>
>         <execution>FIFO</execution>
>         <throttle>60</throttle>
>     </controls>
>     <datasets>
>         <dataset name="inputData" frequency="${coord:minutes(1)}" initial-instance="2009-02-01T00:00Z"
timezone="UTC">
>             <uri-template>hdfs://192.168.138.182:8020/tmp/falcon-regression/NewPrismProcessUpdateTest/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}</uri-template>
>             <done-flag></done-flag>
>         </dataset>
>         <dataset name="inputData2" frequency="${coord:minutes(1)}" initial-instance="2009-02-01T00:00Z"
timezone="UTC">
>             <uri-template>hdfs://192.168.138.182:8020/tmp/falcon-regression/NewPrismProcessUpdateTest/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}</uri-template>
>             <done-flag></done-flag>
>         </dataset>
> {code}
> most of the regression NewPrismProcessUpdateTest is failing coz of this.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message