activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hiram Chirino" <hi...@hiramchirino.com>
Subject Re: Activemq 4.0.2 - stops sending message after some time
Date Wed, 08 Nov 2006 00:24:43 GMT
Hi Sandeep,

Could you post a memory snapshot of the broker from a tool like
jprofiler or yourkit?  That would help folks help you identify what is
holding the memory.

Also if you can simplify your use case down to 1 simple unit test it
would help folks be able to reproduce your problem.

On 11/7/06, Sandeep Chayapathi <sandeep@wssource.com> wrote:
> Update:
>
>  I was using Stomp protocol for both the Publishers and Consumers. A
> check with jconsole showed me that there were around 20,000 threads. To
> test it further, I modified my stomp library (written in perl), to close
> connection and re-connect, every 100 messages. Activemq, consistently
> stopped responding, and the memory used was around 400MB (with another
> 500 MB of physical memory free) and the number of threads were 20,000.
> On selecting the MBean tab in jconsole, I got java.net.connection exception.
>
>  For the next test, I used the ProducerTool (bundled with the activemq
> binaries). The only change was that the ProducerTool would run in a
> while(true){} loop. The idea was to see if/when activemq would break.
> Again, there were 8 Producers and 3 Consuemers (of which, one of them
> was the bundled ConsumerTool and the remaining used Stomp). Around
> 400,000th message (roughly the same number on all 8 processes), the
> jconsole showed non-heap memory used as around 400MB and the number of
> threads at 200, when activemq stopped responding. I had enabled DEBUG
> messages, but no error message were reported in the log. I was able to
> use jconsole to look into the enqueue and dequeue count.
>
> This scenario can be duplicated using the bundled ProducerTool and
> ConsumerTool. I guess the average message/second was 150 per Producer
> process. Also a live environment can easily have 8 or more producers
> (although not with such a high message/second rate), also this set of
> tests show me that the Stomp implementation is not as stable as desired.
>
> Does anyone here use activemq on a production server, if so under what
> load and what is the availability of the activemq service ? How can I
> setup a stable activemq server in a high availability manner. Thanks
>
> - Sandeep
>
> Sandeep Chayapathi wrote:
> > Hi all,
> >
> >  Recently I noticed that the activemq would stop sending message to
> > the consumers after some time. Here is a unscientific test to
> > duplicate this issue:
> > The test involved creating around 7 durable subscriptions and
> > ctrl+c'ing them. Once the durable subscriptions were registered,
> > around 8 publisher processes were started, each process wrote a
> > 'fortune cookie' , in an infiniteloop, to the activemq server. In
> > parallel,  3 subscriber processes were started. The 3 subscribers were
> > randomly stopped and started, cyclically. At about 15000 messages (on
> > each of the 8 write processes), activemq stopped sending the message
> > to to the subscribers and the publisher processes hung on "syswrite"
> > to activemq.  Then another publisher process was started, activemq
> > actually accepted messages, but none of the subscribers received it.
> >
> > The next test involved, using mysql as the data store (instead of the
> > default journalled db). The same steps were followed and at about
> > 18000 messages (on each of the 8 processes), activemq stopped sending
> > messages to the subscribers.
> >
> > Im using activemq 4.0.2. Any suggestions/help is appreciated. Thanks
> >
> > - Sandeep
> >
>
>


-- 
Regards,
Hiram

Blog: http://hiramchirino.com

Mime
View raw message