camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hendley, Sam" <Sam.Hend...@sensus.com>
Subject Failover JMS and Camel DefaultTimeoutManager failures
Date Thu, 26 Sep 2013 15:02:07 GMT
Hello all.

I am working with a large application written in camel. We have recently started noticing
(it may have been occurring for a while) that in some cases a sort of death spiral occurs.

Out setup:
Camel 2.10.1 (unfortunately we can't upgrade)
ActiveMQ 5.6.0 (unfortunately we can't upgrade)

As far as I can tell this is what is occurring:

·         Something puts excess load on the system and slows down our application

·         Some ReplyManagerTimeoutChecks start failing

·         It starts doing excessive amounts of work dealing with timeouts which delays the
processing of the work causing more timeouts

o   Using a sampler it appears most the time is spent in DefaultTimeoutMap.purge()

§  There is a .size() call that is very expensive which is discarded unless trace level logging
is turned on (this bug still exists in the current camel code)

·         The activemq connections start timing out and disconnecting from the broker side

·         This appears to cause the camel routes to spawn a new set of TimeoutChecker and
ReplyManager threads for each of the routes which are all

Has anyone seen issue like this? Any advice on what else I should be looking for?

Thanks

Sam Hendley

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message