tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Philip Wigg" <p...@philipwigg.co.uk>
Subject Tomcat running out of threads.
Date Mon, 21 Apr 2008 14:39:45 GMT
Hi,

I have several Tomcat web applications that run in separate JVMs. I
have one Apache instance that connects to the Tomcat instances via
mod_jk.

I am using:-

RHEL ES 4.0
Java 1.5.0_14
mod_jk-1.2.25-httpd-2.0.59.so
Tomcat 6.0.14.

Usually the sites work fine and are very responsive. The problem is
that I am intermittently seeing the following error in catalina.out:-

SEVERE: All threads (200) are currently busy, waiting. Increase
maxThreads (200) or check the servlet status

Tomcat never recovers from this needs to be restarted. The error does
seem to occur more often, but not exclusively, during periods of heavy
traffic. The thing is that I can see that even during very busy times
the number of busy threads does not normally exceed about 50. It seems
that sometimes the number of busy threads spikes very rapidly and the
system gets into some kind of state whereby threads are not released
for some reason and the system just runs out within about 10 minutes.
I tried increasing the maxThreads to 500, but it still ran out when
this problem occurred.

I am a sysadmin but the developers say that they don't think it is
related to code and that the three applications are different in the
way that they're written.

I have taken a thread dump after this problem occurs, and the threads
are all in the following state:-

"TP-Processor191" daemon prio=1 tid=0x0000002b698b8000 nid=0x3a39
runnable [0x000000004d727000..0x000000004d727bb0]
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:129)
	at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
	at java.io.BufferedInputStream.read1(BufferedInputStream.java:256)
	at java.io.BufferedInputStream.read(BufferedInputStream.java:313)
	- locked <0x0000002adf95e878> (a java.io.BufferedInputStream)
	at org.apache.jk.common.ChannelSocket.read(ChannelSocket.java:620)
	at org.apache.jk.common.ChannelSocket.receive(ChannelSocket.java:558)
	at org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:685)
	at org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:889)
	at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:690)
	at java.lang.Thread.run(Thread.java:595)

I have a very simple workers.properties file which looks like so:-

worker.list=rt-worker,fs-worker,at-worker,tg-worker,gw-worker,gf-worker,pp-worker,bm-worker

worker.rt-worker.type=ajp13
worker.rt-worker.host=localhost
worker.rt-worker.port=8300

worker.fs-worker.type=ajp13
worker.fs-worker.host=localhost
worker.fs-worker.port=8301

[...] (other workers defined as above)

I was wondering if anyone had experienced similar issues or if anyone
could provide some guidance on how to investigate the cause. We've
tried to replicate the issue in our load-testing environment but we've
had no luck so far. Please let me know if there's any other
information you'd like me to provide.

Kind regards,
Philip Wigg.

---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Mime
View raw message