groovy-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alain Stalder <astal...@span.ch>
Subject Re: Improve Groovy class loading performance and memory management
Date Sun, 15 May 2016 08:39:00 GMT
Thanks, that clarifies a lot to me, especially SoftReference.

So with Groovy it is only realistic to have GC of classes (and attached 
ClassInfo) kick in once a limit on Metaspace/PermGen (or Heap) is 
reached - fine with me, no point to try to "outrun the bear"... :)

A general question (current implementation and most likely APIs to keep 
aside): Why does ClassInfo need a reference to the class? To me the use 
case would be that you have an Groovy object or a Groovy class and want 
to do something with it (call a static or instance method, for example), 
so you only need to find ClassInfo from the class and then maybe pass 
the class temporarily just for doing things, but don't need it a 
reference back from ClassInfo.

I presume I am simply overlooking something rather obvious, but what 
would be the use case(s)? Or maybe historical reasons?

I have updated ClassGCTester ("version" 2.0.0) such that the class path 
for the URLClassLoader can be specified in more detail.

https://github.com/jexler/classgc
--
Usage: java [java-args] ClassGCTester -cp <class-path> -parent 
[tester|null] -classes <classes> [-wait]

   -cp:      Class path for URLClassLoader that will load the test classes.
             Directories and JARs separated by ':' or ';'.
             Example: .:classes/:libs/groovy-2.4.6.jar
   -parent:  Parent class loader for URLClassLoader that will load the 
test classes:
             'tester': ClassGCTester.class.getClassLoader() - i.e. same 
as tester
             'null':   null - i.e. no parent
   -classes: Test classes to load from the URLClassLoader in a loop.
             Fully qualified class names separated by ':' or ';'.
             Example: SampleInDefaultPackage:net.sample.Sample
   -wait:    Optional, whether to wait for key pressed before starting 
the test.
             Allows to attach external tools from the start.
             (Like 'jvisualvm' or 'jstat -gc <interval-ms> 
<number-of-iterations>' etc.)
             Note that this tool usually prints out its PID for convenience.
--

This allows, for example, two produce two of the known 
"OutOfMemoryError: Metaspace|PermGen" issues with Groovy 2.4.6, as follows.

Directory Structure:
- GroovyGCTester.class
- groovy-2.4.6.jar
- filling/GroovyFilling.class ("42" compiled, for example, but could be 
practically any Groovy class)

If you load Groovy only once with the class loader of ClassGCTester, and 
set groovy.use.classvalue=false (the default) you get the error:

$ java -XX:MaxMetaspaceSize=64m -Xmx512m -Dgroovy.use.classvalue=false 
-cp .:groovy-2.4.6.jar ClassGCTester -cp filling/ -parent tester 
-classes GroovyFilling
--
Secs Test classes              Metaspace/PermGen Heap   Load time Create 
time
        #loaded  #remaining        used committed       used 
committed     average     average
[...]
   11      8107        8107      38.3m      55.1m     293.8m 455.5m     
0.433ms     0.909ms
   12      8579        8579      40.1m      57.9m     298.0m 455.5m     
0.581ms     0.909ms
   13      8665        8665      40.4m      58.5m     309.5m 455.5m     
0.577ms     0.908ms
   14      9423        9423      43.3m      63.0m     327.0m 455.5m     
0.545ms     0.940ms
Exception in thread "main" java.lang.OutOfMemoryError: Metaspace
[...]
--

With true GC kicks in at the limit of 64m:
--
Secs Test classes              Metaspace/PermGen Heap   Load time Create 
time
        #loaded  #remaining        used committed       used 
committed     average     average
[...]
   12      8579        8579      40.0m      57.9m     300.7m 455.5m     
0.485ms     0.915ms
   13      9424        9424      43.3m      62.8m     330.0m 455.5m     
0.494ms     0.907ms
   14      9670          19       7.3m      30.0m      21.0m 367.0m     
0.527ms     0.908ms
   15     10899        1248      12.0m      31.3m     103.2m 367.0m     
0.484ms     0.880ms
[...]
--

If you instead load Groovy with the same URLClassLoader that loads 
GroovyFilling each time, it passes with groovy.use.classvalue=false (the 
default):

$ java -XX:MaxMetaspaceSize=64m -Xmx512m -Dgroovy.use.classvalue=false 
-cp . ClassGCTester -cp groovy-2.4.6.jar:filling/ -parent null -classes 
GroovyFilling
--
Secs Test classes              Metaspace/PermGen Heap   Load time Create 
time
        #loaded  #remaining        used committed       used 
committed     average     average
[...]
    1         9           9      22.4m      23.1m      26.9m 242.0m     
1.622ms   112.702ms
    2        18          18      38.1m      39.4m      47.9m 308.0m     
1.419ms   112.346ms
    3        27          27      53.7m      55.4m     112.4m 308.0m     
1.367ms   111.185ms
    4        36           5      14.0m      28.0m      41.5m 457.0m     
1.315ms   111.633ms
    5        47          16      33.0m      37.4m      49.0m 474.5m     
1.242ms   106.555ms
[...]
--

Whereas you get the error with true:
--
[...]
Secs Test classes              Metaspace/PermGen Heap   Load time Create 
time
        #loaded  #remaining        used committed       used 
committed     average     average
    0         1           1       8.0m       8.5m      17.2m 245.5m     
2.158ms   119.715ms
    1         9           9      22.3m      23.0m      24.2m 239.5m     
1.596ms   109.466ms
    2        18          18      37.8m      39.3m      46.2m 302.0m     
1.371ms   109.369ms
    3        27          27      53.3m      55.3m     109.8m 302.0m     
1.388ms   109.633ms
Exception in thread "main"
Exception: java.lang.OutOfMemoryError thrown from the 
UncaughtExceptionHandler in thread "main"
--

For completeness, ClassGCTester was compiled with Java 6 for Java 6, 
GroovyFiller with Groovy 2.4.6 (on Java 8, to Java 5) and the tests were 
run with Java 8 (Oracle JDK, Mac).

Alain













Mime
View raw message