tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <Mark_Desp...@McAfee.com>
Subject RE: Headstart on "Resolving OOM-PermGen errors on webapp reload"
Date Tue, 21 Apr 2009 22:17:07 GMT
Hi Ken (and Mark),

Different "Mark" here... I'm new to this mailing list and am not a Tomcat developer.  Forgive
me if I'm interrupting your thread, but I am very interested in this topic, since I've spent
a fair amount of time debugging OOM-PermGen errors within Tomcat (5.5.x).  I would be interested
in hearing any tips or tricks regarding OM-PermGen, too, and am happy to share my experiences
regarding this.  

None of the issues I've looked into have never been attributed to Tomcat.  They have all been
application issues related to an inability to garbage collect a webapp's classloader.  This
root causes were split between one of the following:

* A webapp registering an object with another object that outlives the webapp and forgetting
to unregister it webapp shutdown.  As a result, that object cannot be garbage collected, which
also prevents that webapp's classloader from being garbage collected.  Examples of things
that outlive a webapp or objects from libraries within Tomcat's shared library folder, Tomcat
itself, and the JRE itself.  For example, if a webapp registers a JDBC driver or a JCA/JCE
provider, it should take care to unregister it on shutdown.

* Instantiating a new Thread somehow (e.g. directly via its constructor, or indirectly via
a thread pool, a Timer, or a ScheduledExecutorService) in response to a request to a webapp.
 This is because, by default, the new Thread inherits the parent thread's context classloader,
which in Tomcat (5.5.x) is set to that webapp's classloader. If that Thread keeps the context
classloader reference and outlives the webapp, the webapp's classloader cannot be garbage
collected.

As an aside, those causes highlight a reason why people should be weary of putting a new JAR
files into one of Tomcat's shared library folders. Doing so requires that a webapp be vigilant
about cleaning up any interactions with that library on shutdown.  Said library also needs
to provide a means to do the cleanup, as well as be sensitive to context classloader issues.


When trying to debug these issues, I typically load things up in a profiler, stop the webapp
and see of its classloader still resides in memory.  When it exists, I use the profiler's
"find the garbage collection root" to help determine why the classloader still resides in
memory and figure out how to fix the issue.  If using JProfiler (and probably other profilers),
I think you have to tell it to go not stop when it reaches the Class object along the reference
path.

To help pro-actively detect webapp classloader garbage collection issues, I've leveraged the
Insane library (a library I found from Netbeans while researching the topic) to write a utility
that searches for webapp classloaders that should have been garbage collected.  Using the
utility in combination with automated tests has been definitely helped catch and diagnose
issues.

Apologies if this wasn't helpful.  Please let me know if I'm wrong, outdated, or if there
is a better way!

Best regards, 

Mark DeSpain
 

-----Original Message-----
From: Ken Bowen [mailto:kbowen@als.com] 
Sent: Tuesday, April 21, 2009 1:33 PM
To: Tomcat Users List
Subject: Headstart on "Resolving OOM-PermGen errors on webapp reload"

Mark,

Any chance we could make a headstart on "Resolving OOM-PermGen errors  
on webapp reload" ??
Perhaps some general pointers, guidance etc. [to help you refine the  
talk in advance :-) ]
The manager app is giving me more & more of:
FAIL - Application at context path /ctx could not be started
FAIL - Encountered exception java.lang.OutOfMemoryError: PermGen space
This is on a Tomcat 6.0.18 (java 1.5) running on a remote Linux CentOS  
5, in a Parallels VM, with httpd + mod_jk, running two versions of the  
app. Total memory available is 432 MB, with a number of non-system,  
non-TC processes, including ActiveMQ  and 9 Postres processes. But  
there appears to be about 200MB of unused memory according to the  
Parallels control panel.
I develop locally on a Mac OS/X 10.5.6 box with 4GBmem using  
(My)Eclipse and Tomcat 6.0.18 directly.  With lots & lots of reloads,  
I'm not surprised that I eventually hit OOM PermGen space in this  
setting, but it happens much less often than on the CentOS box (of  
course 4GB != 432MB).
Thanks...Ken
Mark Thomas wrote:
> I was at a conference recently and rather rashly made the statement  
> "OOM
> PermGen errors on reload are an application issue, not a Tomcat one".
> Several people came up to me afterwards with variations on a theme of
> "Our app has OOM on reload - put your money where your mouth is and  
> show
> us where we have gone wrong"
>
> Having helped several people track down the causes of these errors  
> (and
> yes they were all application issues) it occurred to me that there may
> be interest in this at ApacheCon. So my suggestion is:
>
> Title:    Resolving OOM-PermGen errors on webapp reload
> Type:     Session
> Abstract: What causes the error. Typical root causes.
>           How to find and fix the root causes. With live demo.
> Audience: Developer/SysAdmin
>
> Thoughts?
>
> Mark



---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Mime
View raw message