groovy-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alain Stalder <astal...@span.ch>
Subject [ANN] Grengine 1.1.0 - Closable for easier/earlier Garbage Collection
Date Wed, 08 Jun 2016 17:16:07 GMT
Grengine is an engine for running and embedding Groovy in a Java VM.

    https://www.grengine.ch/

Besides the new logo, there is just one new feature:

     Grengine and its loaders etc. can be explicitly *closed*
     which frees Groovy/Java metadata for all(!) classes that
     were loaded with the respective instance.

In practice this has the following effect after closing:

- Oracle Java 8 on Unix and Windows (and apparently Oracle Java 6+7 on 
Windows):
   Groovy classes are garbage collected while the VM is running normally,
   i.e. already before any limit on PermGen/Metaspace or Heap is reached.
- For other VMs, classes are garbage collected when such a limit is reached
   - but - then it is fast(!); it was not noticable in tests, unlike when
    metadata was still present, where a VM could easily hang for seconds...
- (No more OutOfMemoryErrors after closing.)
- (Works with Groovy 1.7.5 to 2.4.7, with and without ClassValue.)

How do you use it:

--
Grengine gren = new Grengine();
gren.run("int x=0; [1,2,3].each { x+=it }; x");
Class clazz = gren.load(...);
Script script = gren.create(...);
...
gren.close();
--

or like this:

--
Grengine gren = new Grengine();
Loader loader1 = gren.newAttachedLoader();
gren.run(loader1, "int x=0; [1,2,3].each { x+=it }; x");
Class clazz = gren.load(loader1, ...);
Script script = gren.create(loader1, ...);
loader1.close();
Loader loader2 = ...
...
loader2.close();
...
gren.close();
--

Since Java 7 you could, of course, also use e.g. a
    "try (Grengine gren = new Grengine()) { ... }"
to close the Grengine etc. instance automatically when it goes out of scope.

How it works (slightly simplified):

Each a time a BytecodeClassLoader loads a Groovy class, it stores a
WeakReference to the class in a ConcurrentLinkedQueue, i.e. fast
concurrent write access and no obstacle to garbage collection.

When closing, the queue is simply polled until it is empty and metadata
for each class that is still found is cleared.

Similarly, Grengine classes that use BytecodeClassLoader instances
keep a WeakReference to each BytecodeClassLoader, again in a
ConcurrentLinkedQueue, which is also polled when closing.

See the default ClassReleaser for what is done exactly:

https://github.com/jexler/grengine/blob/master/src/main/java/ch/grengine/load/DefaultClassReleaser.java

Namely, there is a call to InvokerHelper.removeClass(clazz) plus via 
reflection
a removal of the ClassInfo from either globalClassSet (Groovy < 2.4.0) 
or from
globalClassValue (Groovy >= 2.4.0).

It is also possible to use a different ClassReleaser implementation, in case
this should break in the future or not work in all cases, etc.
(Of course, normally I would also aim to fix the DefaultClassReleaser in a
new Grengine release, such that it would then work in all cases.)

Would be nice if Groovy 2.5 or so would offer an official public cleanup
function for metadata.

Could/should Groovy offer a similar closing mechanism for
GroovyClassLoader, GroovyShell, GroovyScriptEngine, etc.?

Maybe, I don't know, I am also not sure if it is as easily possible there,
but that might be the case...

Some limitations of the current approach in Grengine:

- Only close when you really don't need any loaded classes any more,
   else the metadata has to be recreated and (I guess?) if some methods
   etc. had been added at runtime, that would be lost.
- Has no effect if you use e.g. a GroovyShell or a ConfigSlurper within
   scripts compiled by Grengine on the classes loaded by these Groovy
   utilities (but only these).
- (Does not cover the "Gradle use case" where Groovy itself is loaded
   and unloaded multiple times, there you probably would have to iterate
   through all classes in globalClassValue or globalClassSet via reflection,
   or something like that...)

Alain











Mime
View raw message