activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sandeep Chayapathi <sand...@wssource.com>
Subject Re: Activemq 4.0.2 - stops sending message after some time
Date Tue, 07 Nov 2006 20:20:32 GMT
Update:

 I was using Stomp protocol for both the Publishers and Consumers. A 
check with jconsole showed me that there were around 20,000 threads. To 
test it further, I modified my stomp library (written in perl), to close 
connection and re-connect, every 100 messages. Activemq, consistently 
stopped responding, and the memory used was around 400MB (with another 
500 MB of physical memory free) and the number of threads were 20,000. 
On selecting the MBean tab in jconsole, I got java.net.connection exception.

 For the next test, I used the ProducerTool (bundled with the activemq 
binaries). The only change was that the ProducerTool would run in a 
while(true){} loop. The idea was to see if/when activemq would break. 
Again, there were 8 Producers and 3 Consuemers (of which, one of them 
was the bundled ConsumerTool and the remaining used Stomp). Around 
400,000th message (roughly the same number on all 8 processes), the 
jconsole showed non-heap memory used as around 400MB and the number of 
threads at 200, when activemq stopped responding. I had enabled DEBUG 
messages, but no error message were reported in the log. I was able to 
use jconsole to look into the enqueue and dequeue count.

This scenario can be duplicated using the bundled ProducerTool and 
ConsumerTool. I guess the average message/second was 150 per Producer 
process. Also a live environment can easily have 8 or more producers 
(although not with such a high message/second rate), also this set of 
tests show me that the Stomp implementation is not as stable as desired.

Does anyone here use activemq on a production server, if so under what 
load and what is the availability of the activemq service ? How can I 
setup a stable activemq server in a high availability manner. Thanks

- Sandeep

Sandeep Chayapathi wrote:
> Hi all,
>
>  Recently I noticed that the activemq would stop sending message to 
> the consumers after some time. Here is a unscientific test to 
> duplicate this issue:
> The test involved creating around 7 durable subscriptions and 
> ctrl+c'ing them. Once the durable subscriptions were registered, 
> around 8 publisher processes were started, each process wrote a 
> 'fortune cookie' , in an infiniteloop, to the activemq server. In 
> parallel,  3 subscriber processes were started. The 3 subscribers were 
> randomly stopped and started, cyclically. At about 15000 messages (on 
> each of the 8 write processes), activemq stopped sending the message 
> to to the subscribers and the publisher processes hung on "syswrite" 
> to activemq.  Then another publisher process was started, activemq 
> actually accepted messages, but none of the subscribers received it.
>
> The next test involved, using mysql as the data store (instead of the 
> default journalled db). The same steps were followed and at about 
> 18000 messages (on each of the 8 processes), activemq stopped sending 
> messages to the subscribers.
>
> Im using activemq 4.0.2. Any suggestions/help is appreciated. Thanks
>
> - Sandeep
>


Mime
View raw message