camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From davedave <nab...@scattered.com.au>
Subject Camels stop without warning
Date Tue, 31 Mar 2015 04:57:09 GMT
I'm running Camel (2.14.1) on ServiceMix (5.4.0), and have 21 routes across
two bundles, including 11 timer producing endpoints.

After around 10 hours (~1000 timer messages) of running, camel grinds to a
halt, and only restarting the bundles resolves the issue. I have placed .log
processors between each point in the route, and all running routes seems to
halt at the same time (at various points in the route), with just a few
rouge exchanges making it a few more hops over the following minutes. No
more timers are fired, and in flight exchanges finish their current process
but never make it to the next step in the route.

There is nothing telltale in the logs, even set to DEBUG, everything just
seems to stop, and I just get checkpoint debugs in the servicemix.log from
activemq, until I restart the bundles. Once the bundles get the stop
instruction, they actually begin routing again, but they also begin shutting
down.

I haven't tried running TRACE yet, as those 1000 messages get split in to
many more thousands, and there'd be hundreds of megs of logs every minute
being produced. And with it failing between 8 and 12 hours, there's a 4 hour
window where it would need to be watched closely.

Connecting to the JMX port, I can't see and deadlocked threads.

Any ideas on how best to best debug this?





--
View this message in context: http://camel.465427.n5.nabble.com/Camels-stop-without-warning-tp5765046.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Mime
View raw message