camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Claus Ibsen <claus.ib...@gmail.com>
Subject Re: Quartz2 problem with AsyncDelayedDelivery
Date Tue, 14 Jul 2015 11:46:21 GMT
Hi

Also dont stop Camel from the same thread you process a message. You
should spin that off to a separate thread, or use controlbus in async
mode etc.

On Mon, Jul 13, 2015 at 11:18 AM, jnupponen <jussi.nupponen@gmail.com> wrote:
> Hi!
>
> I've run into a problem with camel-quartz2 (at least on 2.13.2 and 2.15.2)
> cron scheduling and AsyncDelayedDelivery-option in the DefaultErrorHandler.
> If I want to shutdown/restart routes during DefaultErrorHandlers
> redeliveries quartz scheduling gets deadlocked and quietly fails all future
> fire times. Without AsyncDelayedDelivery-option this problem does not occur.
> I'm running my routes in bundles in Karaf 3.0.3 but the problem also occurs
> when running the code on jvm (at least on 1.7) without Karaf.
>
> I have configured quite long redelivery times for my routes (e.g. 8 times in
> 2 hour interval) but at times I would need to be able to restart bundles
> running in Karaf even when there would be redeliveries still left (e.g. when
> installing new version of the bundle).
>
> This seems like a bug to me but what's your take on this? I've attached few
> lines of log and some example route code to the end of this message.
>
> Best regards,
> - Jussi
>
>
>
> Logs on org.quartz debug level show every few seconds:
> [artzScheduler-camel-1_Worker-2] SimpleThreadPool               DEBUG
> Waiting for thread DefaultQuartzScheduler-camel-1_Worker-2 to shut down
>
> and jstack shows for DefaultQuartzScheduler-camel-1_Worker-2:
> "DefaultQuartzScheduler-camel-1_Worker-2" prio=10 tid=0x00007fd2f8201000
> nid=0x2a16 in Object.wait() [0x00007fd3039dc000]
>    java.lang.Thread.State: TIMED_WAITING (on object monitor)
>         at java.lang.Object.wait(Native Method)
>         - waiting on <0x00000007ae410420> (a java.lang.Object)
>         at org.quartz.simpl.SimpleThreadPool.shutdown(SimpleThreadPool.java:371)
>         - locked <0x00000007ae410420> (a java.lang.Object)
>         at org.quartz.core.QuartzScheduler.shutdown(QuartzScheduler.java:740)
>         at org.quartz.impl.StdScheduler.shutdown(StdScheduler.java:206)
>         at
> org.apache.camel.component.quartz2.QuartzComponent.doStop(QuartzComponent.java:352)
>         at org.apache.camel.support.ServiceSupport.stop(ServiceSupport.java:102)
>         at org.apache.camel.util.ServiceHelper.stopService(ServiceHelper.java:141)
>         at
> org.apache.camel.util.ServiceHelper.stopAndShutdownService(ServiceHelper.java:204)
>         at
> org.apache.camel.impl.DefaultCamelContext.shutdownServices(DefaultCamelContext.java:2827)
>         at
> org.apache.camel.impl.DefaultCamelContext.shutdownServices(DefaultCamelContext.java:2852)
>         at
> org.apache.camel.impl.DefaultCamelContext.shutdownServices(DefaultCamelContext.java:2840)
>         at
> org.apache.camel.impl.DefaultCamelContext.doStop(DefaultCamelContext.java:2704)
>         - locked <0x00000007ad1abd88> (a org.apache.camel.impl.DefaultCamelContext)
>         at org.apache.camel.support.ServiceSupport.stop(ServiceSupport.java:102)
>         at
> com.example.route.MyAsyncDelayedDeliveryTesting$1.process(MyAsyncDelayedDeliveryTesting.java:43)
>         at
> org.apache.camel.processor.DelegateSyncProcessor.process(DelegateSyncProcessor.java:63)
>         at
> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:77)
>         at
> org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:448)
>         at
> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
>         at
> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
>         at
> org.apache.camel.processor.loadbalancer.QueueLoadBalancer.process(QueueLoadBalancer.java:44)
>         at
> org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:109)
>         at
> org.apache.camel.processor.loadbalancer.LoadBalancerSupport.process(LoadBalancerSupport.java:87)
>         at org.apache.camel.component.quartz2.CamelJob.execute(CamelJob.java:56)
>         at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
>         at
> org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573)
>         - locked <0x00000007ae539450> (a java.lang.Object)
>
>
> Example of this situation looks like this:
>
> package com.example.route;
>
> import static org.apache.camel.LoggingLevel.ERROR;
> import static org.apache.camel.LoggingLevel.WARN;
>
> import org.apache.camel.Exchange;
> import org.apache.camel.Processor;
> import org.apache.camel.builder.RouteBuilder;
> import org.joda.time.DateTime;
>
> public class MyAsyncDelayedDeliveryTesting extends RouteBuilder {
>
>
>         @Override
>         public void configure() throws Exception {
>
>                 // Set lower timeout for convenience (default 300 seconds).
>                 getContext().getShutdownStrategy().setTimeout(10);
>
>                 // Start some route1 cron scheduled.
>                 DateTime now = DateTime.now().plusSeconds(2);
>                 from("quartz2://my_route_quartz_timer?cron="
>                                 +now.getSecondOfMinute()+"+"
>                                 +now.getMinuteOfHour()+"+"
>                                 +now.getHourOfDay()+"+*+*+?")
>                 .id("scheduling1")
>                 .to("direct://my-route");
>
>
>                 // Start other route that shutdowns the context thus shutting down the
> route1.
>                 // Simulate the situation where you want to restart your route when there
> are still
>                 // ongoing redelivery tasks.
>                 now = DateTime.now().plusSeconds(4);
>                 from("quartz2://my_shutdown_route_quartz_timer?cron="
>                                 +now.getSecondOfMinute()+"+"
>                                 +now.getMinuteOfHour()+"+"
>                                 +now.getHourOfDay()+"+*+*+?")
>                 .id("scheduling2")
>                 .process(new Processor() {
>
>                         @Override
>                         public void process(Exchange arg0) throws Exception {
>                                 getContext().stop();
>
>                         }
>                 });
>
>                 // Do some logic that causes exception and redeliveries for 1 minute.
>                 from("direct://my-route")
>                 .id("route1")
>                 .errorHandler(defaultErrorHandler()
>                         .maximumRedeliveries(10)
>                         .redeliveryDelay(1000*6)
>                         // asyncDelayedRedelivery causes the problem.
>                         .asyncDelayedRedelivery()
>                         .retryAttemptedLogLevel(WARN)
>                         .retriesExhaustedLogLevel(ERROR))
>
>                 .process(new Processor() {
>
>                         @Override
>                         public void process(Exchange exchange) throws Exception {
>                                 throw new RuntimeException("Error");
>                         }
>                 });
>         }
> }
>
> Full example is visible here:
> https://github.com/jnupponen/camel-quartz2-asyncDelayedDelivery-problem
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Quartz2-problem-with-AsyncDelayedDelivery-tp5769210.html
> Sent from the Camel - Users mailing list archive at Nabble.com.



-- 
Claus Ibsen
-----------------
Red Hat, Inc.
Email: cibsen@redhat.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen
hawtio: http://hawt.io/
fabric8: http://fabric8.io/

Mime
View raw message