groovy-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jochen Kemnade (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (GROOVY-6655) GroovyClassLoader parallelLockMap memory leak on Java 7
Date Thu, 27 Jul 2017 08:38:01 GMT

    [ https://issues.apache.org/jira/browse/GROOVY-6655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16102933#comment-16102933
] 

Jochen Kemnade commented on GROOVY-6655:
----------------------------------------

Adding a custom class loader that "blocks" {{loadClass}} does not solve the issue completely,
you'd also have to do something about the script's {{MetaClass}}. When that is created, the
{{defineClass}} goes all the way through to the root loader and there, {{defineClass}} calls
{{loadClass}} again (via a native method). I really don't want to call reflection to the rescue
to fiddle around with the root loader's {{parallelLockMap}}, but I fear that it might be the
only "solution".

> GroovyClassLoader parallelLockMap memory leak on Java 7
> -------------------------------------------------------
>
>                 Key: GROOVY-6655
>                 URL: https://issues.apache.org/jira/browse/GROOVY-6655
>             Project: Groovy
>          Issue Type: Bug
>          Components: groovy-runtime
>    Affects Versions: 2.2.2
>         Environment: Java 7
>            Reporter: Henri Pihkala
>              Labels: classloader, java, leak, memory, metaprogramming
>
> Java 7 introduced parallel classloading. There is a map in {{java.lang.ClassLoader}}
called {{parallelLockMap}}. This map holds String keys for class names and the associated
Objects used to synchronize on for each class.
> Now, I found that dynamically parsing (perhaps also loading?) and unloading classes with
{{GroovyClassLoader}} leaves obsolete keys in this map. Any time {{java.lang.ClassLoader#loadClass(..)}}
is called, an entry is created for that class name if it does not exist. 
> In the default Java delegating classloader paradigm the delegate classloader is asked
for a class first, so a call to the root loader's {{java.lang.ClassLoader#loadClass(..)}}
is made for a whole lot of classes, including Groovy metaclasses, polluting the map with keys.
> !https://dl.dropboxusercontent.com/u/4655232/parallelLockMapMemoryLeak.png!
> The entries that remain in the map even after the class and associated metaclasses themselves
have been unloaded have the following two types of keys:
> {{"groovy.runtime.metaclass.MyDynamicallyLoadedClassNameMetaClass"}}
> {{"MyDynamicallyLoadedClassNameBeanInfo"}}
> A key for the class name itself, ie. {{MyDynamicallyLoadedClassName}}, does not appear
in the map. I am assuming it is never inserted there, but instead defined by {{GroovyClassLoader}}
without asking the parent first. But these metaclass keys *do* leak into the map, eventually
exhausting heap memory even if the classes themselves get nicely unloaded (from PermGen) when
the {{GroovyClassLoader}} is GC'ed.
> Here's a very simple test script that will eventually run out of memory (it does take
some time):
> {code:title=MemoryLeakTest.groovy}
>     String newClass = "class CLASSNAME {}"
>     while (true) {
>         GroovyClassLoader gcl = new GroovyClassLoader()
>         Class clazz = gcl.parseClass(newClass.replace("CLASSNAME", "NewClass"+System.nanoTime()))
>         clazz.newInstance()
>     }
> {code}
> Be sure to run JVM with
> {noformat}
> -XX:+CMSClassUnloadingEnabled -XX:+CMSPermGenSweepingEnabled
> {noformat}
> to make sure you run out of *heap* memory and not *PermGen* space.
> One fix for this would be to add a check to {{GroovyClassLoader}} that would prevent
it from delegating to the parent first if the class to be loaded is a metaclass of a class
loaded by that classloader. I am definitely no expert in this so you might come up with a
much better idea.
> This has been tested on Groovy 2.0.5 and 2.2.2 and I expect that it affects a lot of
versions running on Java 7.
> Sure, it doesn't leak memory at a very fast pace, only a {{ConcurrentHashMap}} entry
for each class loaded and unloaded. I can see how in many applications this is not a major
problem, but please consider for example a server that automatically checks students' programming
assignments or such. Please do change the issue priority if appropriate.
> !https://dl.dropboxusercontent.com/u/4655232/parallelLockMapMemoryLeak2.png!



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message