tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christopher Schultz <ch...@christopherschultz.net>
Subject Re: a lot of threads waiting for a ThreadPool monitor
Date Fri, 13 Jul 2012 13:24:50 GMT
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Alexander,

On 7/13/12 4:34 AM, Alexander Shutyaev wrote:
> I've encountered a problem with an embedded tomcat.

That sound serious.

> The thread dump shows a lot of threads (106 out of 722 total) are
> blocked waiting for a monitor 
> org.apache.tomcat.util.threads.ThreadPool.

Okay. You have maxThreads=1000. Is 106 an unusual number to be idling
in the pool?

> The locking thread is itself blocked on a monitor
> (0x00007f33e30ef000) but I can't find this monitor in the dump
> which is a little bit confusing.

I would have looked for that monitor, but you didn't provide the full
dump, so I'm just as confused as you.

> I thought of a deadlock, but there are no deadlocks detected in the
> dump. My version, config and thread stacktraces are below. Can
> someone help identifying the source of the problem?

Threads in a BLOCKED state are not a problem. Can you maybe describe
your problem in a different way? Has Tomcat stopped responding to
requests? Has CPU usage spiked such that Tomcat/JVM are consuming the
entire CPU resources?

> 5.5.23

Upgrade. I would add something like 1.5.12 to that version up there if
you could. If you are scared of changing major revisions, then try
adding 0.0.7 instead. Note that embedding Tomcat 7 is deliriously
simple when compared to the complex work of embedding past versions:
you may significantly simplify your own code by upgrading and making
the appropriate changes.

> *server.xml*
> 
> <Server port="8005" shutdown="SHUTDOWN"> <GlobalNamingResources /> 
> <Service name="Catalina"> <Connector port="9090"
> maxHttpHeaderSize="8192" maxThreads="1000"

With maxThreads=1000, you might benefit from ... well, at this point
5+ years of performance and API improvements, including NIO connectors
which might allow you to accept more connections while utilizing fewer
threads.

> minSpareThreads="25" maxSpareThreads="75"

maxThreads=1000 maxSpareThreads=75? I guess you are expecting a lot of
thread-pool thrashing.

> enableLookups="false" redirectPort="8443" acceptCount="100" 
> connectionTimeout="20000" disableUploadTimeout="true"
> maxPostSize="0" />

No POSTs? That's odd.

> <Engine name="Catalina" defaultHost="localhost"> <Host
> name="localhost" appBase="webapps" unpackWARs="false" 
> autoDeploy="false" xmlValidation="false" xmlNamespaceAware="false"
> /> </Engine> </Service> </Server>
> 
> *Locking thread stacktrace*
> 
> "http-9090-Processor755" daemon prio=10 tid=0x00007f33fde43000
> nid=0x5ca8 waiting for monitor entry [0x00007f33e30ef000] 
> java.lang.Thread.State: BLOCKED (on object monitor) at
> org.apache.tomcat.util.threads.ThreadPool. 
> returnController(ThreadPool.java:440) - locked <0x0000000685e71038>
> (a org.apache.tomcat.util. threads.ThreadPool) at
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run( 
> ThreadPool.java:712) at java.lang.Thread.run(Thread.java:679)
> 
> *Example of a waiting thread stacktrace*
> 
> "http-9090-Processor504" daemon prio=10 tid=0x000000000432b800
> nid=0x5329 waiting for monitor entry [0x00007f3400218000] 
> java.lang.Thread.State: BLOCKED (on object monitor) at
> org.apache.tomcat.util.threads.ThreadPool. 
> returnController(ThreadPool.java:440) - waiting to lock
> <0x0000000685e71038> (a org.apache.tomcat.util. 
> threads.ThreadPool) at
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run( 
> ThreadPool.java:712) at java.lang.Thread.run(Thread.java:679)
> 

ThreadPool.returnController is called when threads are being returned
to the pool. Do these two threads stay stuck this way if you take
several thread dumps over a few seconds?

- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAlAAIaIACgkQ9CaO5/Lv0PBnEwCgrIChmJrfQUBCCyUQTY7N7BXR
p1cAoMCi1VPWXsEQUPyoN2E8k+rZq2gG
=/Ggc
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Mime
View raw message