Mailing-List: contact tomcat-user-help@jakarta.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "Tomcat Users List" <tomcat-user@jakarta.apache.org>
Received-SPF: pass (hermes.apache.org: local policy)
Message-ID: <41E3CBC1.6040604@fernuni-hagen.de>
Date: Tue, 11 Jan 2005 13:51:13 +0100
From: Oliver Schoenwald <oliver.schoenwald@fernuni-hagen.de>
User-Agent: Mozilla Thunderbird 1.0 (X11/20041206)
MIME-Version: 1.0
To: Oliver Schoenwald <oliver.schoenwald@fernuni-hagen.de>
CC: tomcat-user@jakarta.apache.org
Subject: Re: Tomcat arbitrarily freezes
References: <41DD394F.8020203@fernuni-hagen.de>
In-Reply-To: <41DD394F.8020203@fernuni-hagen.de>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Hi!

five days after my first question and no answer in sight. To bad.
So far, we have switched from JDK 1.4.2 to JDK 1.5 and from Tomcat 
5.0.27 to 5.5.4 and the problem persists.
However, while under JDK 1.4.2 it was always the thread with id #2 under 
JDK 1.5
it is still the thread "VM Thread", but now under id #4.

Using the better debugging features of JDK 1.5 (jstack, jmap) we could 
pin down the problem in more detail:
whenever the problem starts, the output of jstack and jmap shows that 
these debugging-tools have
more and more problems to give information about certain memory areas. 
When the problem reaches its
climax, a bug number of memory areas are marked by jstack with an "Error 
occurred during stack walking:"-Message
or by jmap with "UnmappedAddressException".

You see, at the moment we are only guessing what problem we have here.

Maybe someone has an idea how to analyze the problem more properly or 
even know how to solve it (without changing some of the components like 
using another servlet container engine or another JVM).

Thank you for any help,

Oliver Schoenwald
University of Hagen
Germany

Oliver Schoenwald wrote:

> Our system configuration is as follows:
>
> SUN Fire 240 with 2 cpus at 1.28 GHz (Sparc-3)
> 8 GB RAM
> Solaris 9 (with actual patches)
>
> All requests are handled via Apache 2.0.52 with mod_ssl and using 
> mod_jk2.
>
> The system works well most of the time, but seemingly randomly the 
> catalina process starts to use up
> more and more cpu performance without actually doing anything useful! 
> Memory usage stays normal/stable,
> so we analyzed the process and found out the following:
>
> - The cpu performance is used up primarily by the thread "VM Thread".
> - This Thread is running nothing else but "synchronization 
> primitives", which where called very fast an in high numbers without 
> reading or writing anything (as if it is in some kind of endless loop 
> of doing nothing).
> - The effect (and an symptome which can be monitored repeatedly) is 
> that the number of threads that the catalina process is using rises 
> with every new request, because no thread gets enough cpu power to 
> come to an end. "VM Thread" is taking away anything after a short 
> period of time.
>
> We monitored the machine with prstat -L to see when the cpu usage of 
> the catalina process rose to a unusual high value (5% at average, 
> rising up to about 60% when VM Thread got into its frenzy). So we 
> could see that one single lwp was going into the high cpu usage.
> As soon as we could see the rise in cpu usage we used pstack to find 
> out which thread is responsible for the lwp-id that was shown by 
> prstat -L.
> Then, we used kill -3 to create a thread-dump of the catalina-process 
> and searched for the thread-number that was stated by pstack.
> That thread-dump showed us that the thread called "VM Thread" seems to 
> be the problematic thread.
>


---------------------------------------------------------------------
To unsubscribe, e-mail: tomcat-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: tomcat-user-help@jakarta.apache.org