activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kevan Miller <kevan.mil...@gmail.com>
Subject Re: AMQ5.0 stable release got hung - urgent help needed to understand what happened.
Date Tue, 04 Mar 2008 17:49:41 GMT

On Mar 4, 2008, at 11:23 AM, Dhawan, Vikram (LNG-DAY) wrote:

> Hi,
>
>
>
> Today I got the following message on my AMQ Server logs, since then  
> the
> server was all stuck and there was no client was able to connect to  
> the
> server.  It broke my whole production system. Can some one please help
> me understand what happened here? There were not too many connections
> open at that time. I have Java openwire and C++ STOMP clients. My C++
> clients connect and disconnect every 15 seconds to look for more work;
> my Java OpenWire clients stay connected and create consumers every
> Minute to poll for the new messages.
>
>
>
>
>
>
>
> Exception in thread "ActiveMQ Transport Server: stomp://localhost: 
> 61619"
> java.lang.OutOfMemoryError: unable to create new native thread
>
>        at java.lang.Thread.start0(Native Method)
>
>        at java.lang.Thread.start(Thread.java:574)
>
>        at
> org.apache.activemq.broker.TransportConnector 
> $1.onAccept(TransportConnec
> tor.java:160)
>
>        at
> org 
> .apache.activemq.transport.tcp.TcpTransportServer.run(TcpTransportSer
> ver.java:213)
>
>        at java.lang.Thread.run(Thread.java:595)

What's the maximum heap size for your server process?

Sounds like you have a Thread-leak... Once you encounter this problem,  
can you create a thread dump for all of the java threads in your  
server process? On a unix system, this would be 'kill -3 <server- 
process-id>'

Each java Thread is going to allocate some of your process memory so  
that the OS can manage the thread. Your process is unable to allocate  
enough memory to manage a new Thread. It's possible that specifying a  
smaller max heapsize will give your process more memory for Threads (a  
strange fix for an OutOfMemoryError). However, if the number of  
Threads is growing in an unbounded manner, then it will simply delay  
the inevitable OOME.

I would guess that your STOMP clients are causing the problem. Each  
connect/disconnect is probably burning a thread. Recommend you ask on  
the ActiveMQ user list to see if this is a known problem. Or, if you  
can alter the behavior of your STOMP clients (i.e. long-lived  
connections), I expect this problem will go away... However, I suspect  
this isn't what you want... ;-)

--kevan

Mime
View raw message