camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hendley, Sam" <Sam.Hend...@sensus.com>
Subject RE: Failover JMS and Camel DefaultTimeoutManager failures
Date Fri, 27 Sep 2013 13:55:13 GMT
Hi Christian, thanks for your interest. 

As I said the size of the application is very large, we have around 150 route builders spread
over 5 or so processes. I can't easily boil down our issue to a few configurations or route
builders. I was just trying to see if issues like this have been encountered before. I am
trying to make a case for upgrading ActiveMQ and Camel but would like to know if the camel
community believes that will clear up this issue.

Thanks

Sam 

-----Original Message-----
From: Christian Posta [mailto:christian.posta@gmail.com] 
Sent: Thursday, September 26, 2013 11:07 AM
To: users@camel.apache.org
Subject: Re: Failover JMS and Camel DefaultTimeoutManager failures

Post your route, your configs, and a description of what it does, who are its clients, etc.
Also link to the bug in the current code.


On Thu, Sep 26, 2013 at 8:02 AM, Hendley, Sam <Sam.Hendley@sensus.com>wrote:

> Hello all.
>
> I am working with a large application written in camel. We have 
> recently started noticing (it may have been occurring for a while) 
> that in some cases a sort of death spiral occurs.
>
> Out setup:
> Camel 2.10.1 (unfortunately we can't upgrade) ActiveMQ 5.6.0 
> (unfortunately we can't upgrade)
>
> As far as I can tell this is what is occurring:
>
> ·         Something puts excess load on the system and slows down our
> application
>
> ·         Some ReplyManagerTimeoutChecks start failing
>
> ·         It starts doing excessive amounts of work dealing with timeouts
> which delays the processing of the work causing more timeouts
>
> o   Using a sampler it appears most the time is spent in
> DefaultTimeoutMap.purge()
>
> §  There is a .size() call that is very expensive which is discarded 
> unless trace level logging is turned on (this bug still exists in the 
> current camel code)
>
> ·         The activemq connections start timing out and disconnecting from
> the broker side
>
> ·         This appears to cause the camel routes to spawn a new set of
> TimeoutChecker and ReplyManager threads for each of the routes which 
> are all
>
> Has anyone seen issue like this? Any advice on what else I should be 
> looking for?
>
> Thanks
>
> Sam Hendley
>



--
*Christian Posta*
http://www.christianposta.com/blog
twitter: @christianposta

Mime
View raw message