falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ajay Yadava (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FALCON-1530) SLAMonitoring API is not honouring delete feature
Date Tue, 13 Oct 2015 05:32:05 GMT

    [ https://issues.apache.org/jira/browse/FALCON-1530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14954420#comment-14954420
] 

Ajay Yadava commented on FALCON-1530:
-------------------------------------

Committed to master.
Commit id: 666a2a2

Committed to 0.8
Commit id: 71806b9

Thanks for reviewing [~bvellanki], [~sowmyaramesh] and [~sandeep.samudrala].

[~sowmyaramesh] I assumed that I answered all your queries satisfactorily, if there are any
concerns please do let me know and I will happily address them in separate JIRA.

> SLAMonitoring API is not honouring delete feature
> -------------------------------------------------
>
>                 Key: FALCON-1530
>                 URL: https://issues.apache.org/jira/browse/FALCON-1530
>             Project: Falcon
>          Issue Type: Bug
>         Environment: QA
>            Reporter: Pragya Mittal
>            Assignee: Ajay Yadava
>             Fix For: 0.8
>
>         Attachments: FALCON-1530.patch
>
>
> In distributed environment, SLA monitoring API gives null response even though there
are instances having SLA miss with EntityNotRegisteredException.
> This happens under following scenario:
> On deleting entity, it gets removed from store but the checkpoint is still present in
list of sla pending instances. Hence, hitting sla monitoring api after deleting entity will
keep on looking for deleted entity which will eventually lead to failure.
> Feed definition is :
> {noformat}
> dataqa@lda01:/mnt/falcon/server/logs$ fef -definition -name FeedSlaMonitoring-raaw-logs16-d9eed56e
> <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
> <feed name="FeedSlaMonitoring-raaw-logs16-d9eed56e" description="clicks log" xmlns="uri:falcon:feed:0.1">
>     <partitions>
>         <partition name="country"/>
>         <partition name="colo"/>
>     </partitions>
>     <frequency>minutes(1)</frequency>
>     <sla slaLow="minutes(1)" slaHigh="minutes(5)"/>
>     <timezone>UTC</timezone>
>     <late-arrival cut-off="hours(6)"/>
>     <clusters>
>         <cluster name="FeedSlaMonitoring-corp-1aa25120">
>             <validity start="2015-10-12T12:24Z" end="2015-10-12T14:24Z"/>
>             <retention limit="days(1000000)" action="delete"/>
>         </cluster>
>     </clusters>
>     <locations>
>         <location type="data" path="/tmp/falcon-regression/FeedSlaMonitoring/input/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}"/>
>         <location type="stats" path="/projects/falcon/clicksStats"/>
>         <location type="meta" path="/projects/falcon/clicksMetaData"/>
>     </locations>
>     <ACL owner="pragya" group="dataqa" permission="*"/>
>     <schema location="/schema/clicks" provider="protobuf"/>
>     <properties>
>         <property name="field1" value="value1"/>
>         <property name="field2" value="value2"/>
>     </properties>
> </feed>
> {noformat}
> Entities present at the moment :
> {noformat}
> dataqa@lda01:/mnt/falcon/server/logs$ fef -list
> 1
> (FEED) FeedSlaMonitoring-raaw-logs16-d9eed56e
> {noformat}
> Sla response is as follows :
> {noformat}
> Mon Oct 12 12:58:47 GMT 2015
> dataqa@lda01:~$ falcon entity -type feed -start 2015-10-12T11:50Z -slaAlert
> No sla miss found!
> Response: ua1/org.apache.falcon.FalconException::javax.ws.rs.WebApplicationException:
javax.xml.bind.UnmarshalException: unexpected element (uri:"", local:"instancesResult"). Expected
elements are <{}instances>,<{}result>
> ua3/Success!
> ua2/Success!
> Request Id: ua1/1833028654@qtp-477289012-21 - 4dbe4e5d-026b-48f8-bb3e-879dfbf502a2
> ua3/319644606@qtp-1334042472-0 - f5c1a05c-e2f9-4502-92aa-7b91947f5b12
> ua2/193086798@qtp-690643456-0 - a30f4143-6d59-464e-a924-ef69430af0e5
> {noformat}
> Sla stack trace is :
> {noformat}
> 2015-10-12 12:26:17,977 Adding feed:FeedSlaMonitoring-raaw-logs16-d9eed56e for monitoring
> 2015-10-12 12:32:09,401 Feed SLA monitoring failed: 
> org.apache.falcon.entity.EntityNotRegisteredException: FeedSlaMonitoring-raaw-logs16-63ed08dd
(FEED) not found
> 	at org.apache.falcon.entity.EntityUtil.getEntity(EntityUtil.java:98)
> 	at org.apache.falcon.service.FeedSLAMonitoringService.checkFeedInstanceAvailability(FeedSLAMonitoringService.java:330)
> 	at org.apache.falcon.service.FeedSLAMonitoringService.checkPendingInstanceAvailability(FeedSLAMonitoringService.java:319)
> 	at org.apache.falcon.service.FeedSLAMonitoringService.access$200(FeedSLAMonitoringService.java:64)
> 	at org.apache.falcon.service.FeedSLAMonitoringService$Monitor.run(FeedSLAMonitoringService.java:256)
> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> 	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> 	at java.lang.Thread.run(Thread.java:745)
> 2015-10-12 12:42:09,402 Feed SLA monitoring failed: 
> org.apache.falcon.entity.EntityNotRegisteredException: FeedSlaMonitoring-raaw-logs16-63ed08dd
(FEED) not found
> 	at org.apache.falcon.entity.EntityUtil.getEntity(EntityUtil.java:98)
> 	at org.apache.falcon.service.FeedSLAMonitoringService.checkFeedInstanceAvailability(FeedSLAMonitoringService.java:330)
> 	at org.apache.falcon.service.FeedSLAMonitoringService.checkPendingInstanceAvailability(FeedSLAMonitoringService.java:319)
> 	at org.apache.falcon.service.FeedSLAMonitoringService.access$200(FeedSLAMonitoringService.java:64)
> 	at org.apache.falcon.service.FeedSLAMonitoringService$Monitor.run(FeedSLAMonitoringService.java:256)
> 	at java.util.concurrent.E
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message