falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ajay Yadava" <ajayn...@gmail.com>
Subject Re: Review Request 38166: Add shutdown hook for Falcon
Date Wed, 09 Sep 2015 07:46:56 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38166/#review98156
-----------------------------------------------------------



rerun/src/main/java/org/apache/falcon/rerun/handler/AbstractRerunConsumer.java (line 54)
<https://reviews.apache.org/r/38166/#comment154440>

    Why not use a volatile boolean and use that for shutting down the thread cleanly instead
of interrupting it and then doing all the error handling?



rerun/src/main/java/org/apache/falcon/rerun/handler/AbstractRerunConsumer.java (line 61)
<https://reviews.apache.org/r/38166/#comment154439>

    It will be better to log the stacktrace as the reason for interrupt might vary.


- Ajay Yadava


On Sept. 9, 2015, 6:23 a.m., sandeep samudrala wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38166/
> -----------------------------------------------------------
> 
> (Updated Sept. 9, 2015, 6:23 a.m.)
> 
> 
> Review request for Falcon.
> 
> 
> Bugs: FALCON-348
>     https://issues.apache.org/jira/browse/FALCON-348
> 
> 
> Repository: falcon-git
> 
> 
> Description
> -------
> 
> A shutdown hook is missing and services are shutdown in a brute force mode. I'd like
to see activemq, titan and such are shutdown gracefully so we don't lose data and see ugly
exceptions in the logs.
> 
> 
> Diffs
> -----
> 
>   prism/src/main/java/org/apache/falcon/Main.java 96e003c 
>   rerun/src/main/java/org/apache/falcon/rerun/handler/AbstractRerunConsumer.java 9ee94c5

>   rerun/src/main/java/org/apache/falcon/rerun/handler/AbstractRerunHandler.java f019737

>   rerun/src/main/java/org/apache/falcon/rerun/handler/LateRerunHandler.java c2cb09e 
>   rerun/src/main/java/org/apache/falcon/rerun/handler/RetryHandler.java c6bc36f 
>   rerun/src/main/java/org/apache/falcon/rerun/queue/ActiveMQueue.java 021e4cc 
>   rerun/src/main/java/org/apache/falcon/rerun/service/LateRunService.java 2bb198b 
> 
> Diff: https://reviews.apache.org/r/38166/diff/
> 
> 
> Testing
> -------
> 
> I have started and stopped the server. Logs look clean and each service gets shutdown
properly. 
> Handled activemq brokerservice and server stops separately.
> 
> Below is how the logs are looking after the patch.
> 
> 2015-09-07 18:08:02,913 INFO  - [Thread-0:] ~ calling shutdown hook (Main:68)
> 2015-09-07 18:08:02,913 INFO  - [Thread-0:] ~ Stopped SocketConnector@0.0.0.0:15000 (log:67)
> 2015-09-07 18:08:02,922 INFO  - [Thread-0:] ~ Destroying service: org.apache.falcon.security.AuthenticationInitializationService
(ServiceInitializer:58)
> 2015-09-07 18:08:02,923 INFO  - [Thread-0:] ~ Service destroyed: org.apache.falcon.security.AuthenticationInitializationService
(ServiceInitializer:65)
> 2015-09-07 18:08:02,923 INFO  - [Thread-0:] ~ Destroying service: org.apache.falcon.workflow.WorkflowJobEndNotificationService
(ServiceInitializer:58)
> 2015-09-07 18:08:02,923 INFO  - [Thread-0:] ~ Service destroyed: org.apache.falcon.workflow.WorkflowJobEndNotificationService
(ServiceInitializer:65)
> 2015-09-07 18:08:02,923 INFO  - [Thread-0:] ~ Destroying service: org.apache.falcon.service.ProcessSubscriberService
(ServiceInitializer:58)
> 2015-09-07 18:08:02,923 INFO  - [Thread-0:] ~ Closing topicSubscriber on topic : FALCON.ENTITY.TOPIC
(JMSMessageConsumer:149)
> 2015-09-07 18:08:02,943 INFO  - [Thread-0:] ~ Attempting to close connection (MessagingUtil:65)
> 2015-09-07 18:08:02,959 INFO  - [Thread-0:] ~ Service destroyed: org.apache.falcon.service.ProcessSubscriberService
(ServiceInitializer:65)
> 2015-09-07 18:08:02,959 INFO  - [Thread-0:] ~ Destroying service: org.apache.falcon.entity.store.ConfigurationStore
(ServiceInitializer:58)
> 2015-09-07 18:08:02,959 INFO  - [Thread-0:] ~ Service destroyed: org.apache.falcon.entity.store.ConfigurationStore
(ServiceInitializer:65)
> 2015-09-07 18:08:02,960 INFO  - [Thread-0:] ~ Destroying service: org.apache.falcon.rerun.service.RetryService
(ServiceInitializer:58)
> 2015-09-07 18:08:02,960 INFO  - [Thread-0:] ~ RetryHandler thread destroyed (RetryService:65)
> 2015-09-07 18:08:02,960 INFO  - [Thread-0:] ~ Service destroyed: org.apache.falcon.rerun.service.RetryService
(ServiceInitializer:65)
> 2015-09-07 18:08:02,960 INFO  - [Thread-0:] ~ Destroying service: org.apache.falcon.rerun.service.LateRunService
(ServiceInitializer:58)
> 2015-09-07 18:08:02,963 INFO  - [Thread-0:] ~ Closing queue for broker=tcp://localhost:61616,
destinationfalcon.late.queue (DelayedQueue:132)
> 2015-09-07 18:08:02,964 INFO  - [Thread-0:] ~ Attempting to close producer (MessagingUtil:76)
> 2015-09-07 18:08:02,965 INFO  - [LaterunHandler:] ~ Rerun handler daemon has been interrupted
(AbstractRerunConsumer:62)
> 2015-09-07 18:08:02,966 INFO  - [Thread-0:] ~ Attempting to close consumer (MessagingUtil:87)
> 2015-09-07 18:08:02,967 INFO  - [Thread-0:] ~ Attempting to close connection (MessagingUtil:65)
> 2015-09-07 18:08:02,978 INFO  - [Thread-0:] ~ Closing queue for broker=tcp://localhost:61616,
destinationfalcon.late.queue (DelayedQueue:132)
> 2015-09-07 18:08:02,980 INFO  - [Thread-0:] ~ Attempting to close producer (MessagingUtil:76)
> 2015-09-07 18:08:02,980 INFO  - [Thread-0:] ~ Attempting to close consumer (MessagingUtil:87)
> 2015-09-07 18:08:02,984 INFO  - [Thread-0:] ~ Attempting to close connection (MessagingUtil:65)
> 2015-09-07 18:08:02,984 INFO  - [Thread-0:] ~ LateRun thread destroyed (LateRunService:70)
> 2015-09-07 18:08:02,984 INFO  - [Thread-0:] ~ Service destroyed: org.apache.falcon.rerun.service.LateRunService
(ServiceInitializer:65)
> 2015-09-07 18:08:02,984 INFO  - [Thread-0:] ~ Destroying service: org.apache.falcon.metadata.MetadataMappingService
(ServiceInitializer:58)
> 2015-09-07 18:08:02,985 INFO  - [Thread-0:] ~ Shutting down graph db (MetadataMappingService:202)
> 2015-09-07 18:08:03,046 INFO  - [Thread-0:] ~ Service destroyed: org.apache.falcon.metadata.MetadataMappingService
(ServiceInitializer:65)
> 2015-09-07 18:08:03,046 INFO  - [Thread-0:] ~ Destroying service: org.apache.falcon.service.LogCleanupService
(ServiceInitializer:58)
> 2015-09-07 18:08:03,046 INFO  - [Thread-0:] ~ Falcon log cleanup service destroyed (LogCleanupService:80)
> 2015-09-07 18:08:03,046 INFO  - [Thread-0:] ~ Service destroyed: org.apache.falcon.service.LogCleanupService
(ServiceInitializer:65)
> 2015-09-07 18:08:03,046 INFO  - [Thread-0:] ~
> ############################################
>          Falcon Server (SHUTDOWN)
> ############################################ (ContextStartupListener:94)
> 2015-09-07 18:08:03,047 INFO  - [Thread-0:] ~ ActiveMQ Message Broker (localhost, ID:groot-54663-1441649273016-0:1)
is shutting down (BrokerService:560)
> 2015-09-07 18:08:03,048 INFO  - [Thread-0:] ~ Connector vm://localhost Stopped (TransportConnector:288)
> 2015-09-07 18:08:03,632 INFO  - [Thread-0:] ~ Connector tcp://groot:61616 Stopped (TransportConnector:288)
> 2015-09-07 18:08:03,641 INFO  - [Thread-0:] ~ JobSchedulerStore:/home/dataqa/sandeep/falcon-0.8-SNAPSHOT/data/localhost/scheduler
stopped (JobSchedulerStore:286)
> 2015-09-07 18:08:03,642 INFO  - [Thread-0:] ~ PListStore:/home/dataqa/sandeep/falcon-0.8-SNAPSHOT/data/localhost/tmp_storage
stopped (PListStore:307)
> 2015-09-07 18:08:03,642 INFO  - [Thread-0:] ~ Stopping async queue tasks (KahaDBStore:206)
> 2015-09-07 18:08:03,643 INFO  - [Thread-0:] ~ Stopping async topic tasks (KahaDBStore:220)
> 2015-09-07 18:08:03,643 INFO  - [Thread-0:] ~ Stopped KahaDB (KahaDBStore:246)
> 2015-09-07 18:08:04,048 INFO  - [Thread-0:] ~ ActiveMQ JMS Message Broker (localhost,
ID:groot-54663-1441649273016-0:1) stopped (BrokerService:628)
> 
> 
> Thanks,
> 
> sandeep samudrala
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message