tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Oliver Schoenwald <oliver.schoenw...@fernuni-hagen.de>
Subject Tomcat arbitrarily freezes
Date Thu, 06 Jan 2005 13:12:47 GMT
Hello,

we are using Tomcat 5.0.27 to provide the eLearning environment of our 
University.

Our system configuration is as follows:

SUN Fire 240 with 2 cpus at 1.28 GHz (Sparc-3)
8 GB RAM
Solaris 9 (with actual patches)

All requests are handled via Apache 2.0.52 with mod_ssl and using mod_jk2.

The system works well most of the time, but seemingly randomly the 
catalina process starts to use up
more and more cpu performance without actually doing anything useful! 
Memory usage stays normal/stable,
so we analyzed the process and found out the following:

- The cpu performance is used up primarily by the thread "VM Thread".
- This Thread is running nothing else but "synchronization primitives", 
which where called very fast an in high numbers without reading or 
writing anything (as if it is in some kind of endless loop of doing 
nothing).
- The effect (and an symptome which can be monitored repeatedly) is that 
the number of threads that the catalina process is using rises with 
every new request, because no thread gets enough cpu power to come to an 
end. "VM Thread" is taking away anything after a short period of time.

To come to this information, we did the following:

We monitored the machine with prstat -L to see when the cpu usage of the 
catalina process rose to a unusual high value (5% at average, rising up 
to about 60% when VM Thread got into its frenzy). So we could see that 
one single lwp was going into the high cpu usage.
As soon as we could see the rise in cpu usage we used pstack to find out 
which thread is responsible for the lwp-id that was shown by prstat -L.
Then, we used kill -3 to create a thread-dump of the catalina-process 
and searched for the thread-number that was stated by pstack.
That thread-dump showed us that the thread called "VM Thread" seems to 
be the problematic thread.

As far as we found out that problems occured several times with Tomcat 5 
for other users in the world. However, so far no one offered an 
explanation what is going wrong there and what could be done to avoid 
that problem.

We now have prepared a monitoring service that kills and restarts the 
catalina engine as soon as the problem occurs, but that solution is not 
a good one.

Any suggestions of how to solve the problem are very, very welcome. If 
you need more information or have any idea how we should analyze the 
situation more properly, I would be happy to hear of it.


Thank you in advance,


Oliver Schoenwald
FernUniversitaet Hagen
Germany



---------------------------------------------------------------------
To unsubscribe, e-mail: tomcat-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: tomcat-user-help@jakarta.apache.org


Mime
View raw message