logging-log4j-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Remko Popma (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (LOG4J2-819) PermGen OutOfMemoryError when reloading webapp on Tomcat 6
Date Sat, 13 Sep 2014 08:10:33 GMT

    [ https://issues.apache.org/jira/browse/LOG4J2-819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14132571#comment-14132571
] 

Remko Popma edited comment on LOG4J2-819 at 9/13/14 8:09 AM:
-------------------------------------------------------------

Costa, thanks for helping with the investigation.

I think this is caused by Tomcat trying to prevent memory leaks. (Ironic isn't it?)
http://wiki.apache.org/tomcat/MemoryLeakProtection#staticClassVariables

{quote}
When an app is stopped, Tomcat (even before 6.0.24) nullifies the value of all static class
variables of classes loaded by the WebAppClassLoader. 
{quote}

So Tomcat, bless its little white cotton socks, causes the very problem it is trying to prevent:
referencing the static fields in CachedClock and CoarseCachedClock will initialize these classes,
which starts the background threads. Once these threads are started, the web app cannot be
unloaded anymore, and voila, we have a memory leak.

I'm sure we are not the first to have this problem because Tomcat provides a workaround. They
suggest to configure a [JRE Memory Leak Prevention Listener|http://tomcat.apache.org/tomcat-6.0-doc/config/listeners.html#JRE_Memory_Leak_Prevention_Listener_-_org.apache.catalina.core.JreMemoryLeakPreventionListener].
This supports a {{classesToInitialize}} attribute where we can specify the classes that should
be loaded by Tomcat's common class loader instead of the webapp's context class loader. 

I tried this, but unfortunately this initialization takes place at Tomcat startup, and with
the log4j-core jar inside the demo.war file and not in Tomcat's lib folder I got a  JreMemoryLeakPreventionListener
ClassNotFoundException "Failed to load class org.apache.logging.log4j.core.util.CachedClock
during Tomcat start to prevent possible memory leaks". So the workaround does not work for
us here.

Potential solutions:
* Modify ClockFactory so that the CachedClock and CoarseCachedClock classes are not loaded
unless the user explicitly requests these clocks, using reflection in ClockFactory.
* Remove CachedClock and CoarseCachedClock from log4j-core altogether.
* A stop() mechanism will not help because there is nobody around to call it (the web app
has already been stopped before these threads are started by Tomcat's cleanup)

Although I suggested it, I'm still a bit on the fence about removing these classes. Remember
that [benchmarks|https://issues.apache.org/jira/browse/LOG4J2-744?focusedCommentId=14077425&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14077425]
showed that System.currentTimeMillis() can take 3+ microseconds on Linux, vs 14 nanoseconds
on Windows: a 220x(!) difference... 

Thoughts?


was (Author: remkop@yahoo.com):
Costa, thanks for helping with the investigation.
I think this is caused by Tomcat trying to prevent memory leaks. (Ironic isn't it?)
http://wiki.apache.org/tomcat/MemoryLeakProtection#staticClassVariables

{quote}
When an app is stopped, Tomcat (even before 6.0.24) nullifies the value of all static class
variables of classes loaded by the WebAppClassLoader. 
{quote}


> PermGen OutOfMemoryError when reloading webapp on Tomcat 6
> ----------------------------------------------------------
>
>                 Key: LOG4J2-819
>                 URL: https://issues.apache.org/jira/browse/LOG4J2-819
>             Project: Log4j 2
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 2.0.2
>         Environment: Tomcat 6
>            Reporter: Costa Theodosiou
>            Priority: Critical
>         Attachments: demo.zip, demorun-tomcat6-with-reload.zip, demorun.zip, gg-log4j2-clocks-interrupts.patch,
gg-log4j2-clocks-v2.patch
>
>
> When reloading an application 3 or 4 times in Tomcat 6, the application crashes with
a "java.lang.OutOfMemoryError: PermGen space" exception.
> After some investigation using the "When all else fails" section of https://wiki.apache.org/tomcat/OutOfMemory
in conjunction with Java VisualVM, I have narrowed down the problem to the Thread created
within org.apache.logging.log4j.core.util.CoarseCachedClock.
> When a Thread is created, it contains a reference to the classloader that it was created
with. In this case, the Thread's contextClassLoader field contains a reference to the WebappClassLoader.
When Tomcat attempts to unload the webapp, the Thread still holds onto this reference which
prevents WebappClassLoader from being freed.
> Perhaps the Log4jServletContextListener (Log4jWebInitializerImpl) can be made to stop
the CoarseCachedClock thread.
> I believe this is not an obvious issue on Tomcat 7 due to https://wiki.apache.org/tomcat/MemoryLeakProtection.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-dev-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-dev-help@logging.apache.org


Mime
View raw message