tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Crowther <peter.crowt...@melandra.com>
Subject Re: Tomcat dies suddenly (was JVM goes away)
Date Wed, 13 Jan 2010 13:49:54 GMT
Very difficult to know what the problem is.  One thing you can now do
(as you've switched to another production server) is to run a memory
test across the "bad" server.  A T110 doesn't use error-correcting
memory, as I recall, so a dodgy bit could cause problems.  Give it a
couple of hours with memtest86+ and you'll at least know whether
you've been chasing phantoms due to a hardware error.

(I'm perhaps biased - I've had memory errors on three low-end servers now)

- Peter

2010/1/13 Carl <carl@etrak-plus.com>:
> From the original posting:
>
> This is a new server, a Dell T110 with a Xeon 3440 processor and 4GB memory.  I have
turned off both the turbo mode and hyperthreading.
>
> The environment:
>
> 64 bit Slackware Linux
>
> java version "1.6.0_17"
> Java(TM) SE Runtime Environment (build 1.6.0_17-b04)
> Java HotSpot(TM) 64-Bit Server VM (build 14.3-b01, mixed mode)
>
> Tomcat: apache-tomcat-6.0.20
>
> These are the current JAVA_OPTS="-Xms1024m -Xmx1024m -XX:PermSize=368m -XX:MaxPermSize=368m"
>
> In the previous posting, I noted that I have observed the memory usage and general performance
with Java VisualVM and have seen nothing strange.  GC seems to be performing well and the
memory rarely gets anywhere near the max.  New information: I thought I was seeing GC as
memory usage was going up and down but in fact it was mostly people coming onto the system
and leaving it.  After several hours, the memory settles to a baseline of about 375MB.  Forced
GC never takes it below that value and the ups and downs from the people coming onto and leaving
the system also returns it to pretty much that value.  The maximum memory used never was
above 700MB for the entire day.
>
> The server runs well, idling along at 2-5% load, except for a quick spike during GC,
serving jsp's, etc. at a reasonable speed.  Without warning and with no tracks in any log
(Tomcat or system) or to the console, the JVM will just go away, disappear.  New information:
The JVM does not just go away but somehow Tomcat shutsdown as the ports used by Tomcat are
closed (pointed out by Konstantin.)  Sometimes, the system will run for a week, sometimes
for only several hours.  Initially, I thought the problem was the turbo or hyperthreading
but, no, the problem persists.
>
> When Tomcat shuts down, the memory that it held is still being held (as seen from top)
but it is nowhere near the machine physical memory.
>
> The application has been running on an older server (Dell 600SC, 32 bit Slackware, 2GB
memory) for several years and, while the application will throw exceptions now and then, it
never crashed.  This lead me to believe the problem had something to do with the 64 bit JVM
but, with without seeing errors anywhere, I can't be certain and don't know what I can do
about it except go back to 32 bit.
>
> New information.
>
> Last evening, I observed the heap and permGen memory usage with Visual JVM.  It was
running around 600MB before I forced a GC and 375MB afterward.  Speed was good.  Memory
usage from top was 2.4GB.  Five minutes later, Tomcat stopped leaving no tracks that I could
find.  The memory usage from top was around 2.4GB.  The memory usage from Visual JVM was
still showing 400MB+ although the Tomcat process was gone.  I restarted Tomcat (did not reboot)
so Tomcat had been shutdown gracefully enough to close the ports (8080, 8443, 443.)  Tomcat
stayed up for less than an hour (under light load) and stopped again.  The memory used according
to top was less than 3GB but I didn't get the exact number.  I restarted it again (no server
reboot) and it ran for the rest of the night (light load) and top was showing 3.3GB for memory
this morning.
>
> I brought up a new server last night and have switched to that server for production
(same Linux, JDK, server.xml, JAVA_OPTS, etc.).  It would seem if the problem is with my
application or the JVM, that the problem will follow me to the new server.
>
> Anyone have any ideas how I might track this problem down?
>
> Thanks,
>
> Carl

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Mime
View raw message