activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arjen van der Meijden <acmmail...@tweakers.net>
Subject Gradually increasing, huge cpu-load when using Stomp+nio
Date Wed, 20 Jan 2010 21:08:17 GMT
Hi List,

I just restarted a activemq 5.3-instance which was causing 1000% cpu 
(the machine has 8 cores, so all where fully working) and a load of 1035.

I'm pretty confident that this load/cpu-usage was caused by using 
stomp+nio rather than 'stomp' (whether it was just nio or not, I don't 
know).

Our usage-pattern is that we have a few queues with a single consumer 
each. The producers are php-web-processes that make a connection using 
stomp, send a few messages to some of the queues and disconnect again 
(which is why we have the transport.closeAsync=false). The heaviest two 
queues receive about 30-50 messages/second during peak hours (and thus, 
ActiveMQ receives about 30-50 connections/second).

We also have a few topics that get messages quite infrequently, from 
similar stomp-producers, but those are consumed with java-based 
applications.

The load/cpu-usage apparently has gradually increased, as can be seen in 
these two graphs:
Cpu-usage
http://tweakers.net/stats/?Action=Generator&Mode=Serverstats&Time=1263919627&Dagen=1&StatsServer=Argus&colServers=CPUUsage

Load average
http://tweakers.net/stats/?Action=Generator&Mode=Serverstats&Time=1263919627&Dagen=1&StatsServer=Argus&colServers=LoadAvg

The increase in load has gone steadily in just a few hours up untill the 
moment where it took the php-processes more than 13 seconds to actually 
connect, send two messages and disconnect (rather than a few milliseconds).

Our activemq.xml is attached, although I already replaced stomp+nio with 
normal stomp. Our ACTIVEM_OPTS is "-Xms2048M -Xmx2048M 
-XX:+UseParallelOldGC -server 
-Dorg.apache.activemq.UseDedicatedTaskRunner=true" (i.e. the default 
with adjusted memory and GC)

The increasing load hasn't appeared yet with the adjusted config that 
uses non-nio connections.

Unfortunately, this is a production server so I can't run tests very 
easily. Nor do I have a problem with not running nio, so this is mainly 
a mail to let you know of this problem, rather than me having a big 
issue with activemq right now :)

Best regards,

Arjen

Mime
View raw message