groovy-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alain Stalder <astal...@span.ch>
Subject Re: Improve Groovy class loading performance and memory management
Date Tue, 17 May 2016 05:53:53 GMT
That looks very good to me :)

I will definitely try out the InvokerHelper.removeClass(clazz) with 
added ClassInfo removal plus Introspector.flushFromCaches(clazz) and see 
if I can get garbage collection before reaching the limit on Metaspace 
or Heap.

And, maybe something like the following could be added to the 
GroovyClassLoader? Thinking aloud:

Assuming the following is true: Any class can only be garbage collected 
once its ClassLoader can be garbage collected, because each class keeps 
a reference to its ClassLoader (so that it can use it to load further 
classes when needed when running methods).

So why not have the GroovyClassLoader keep a set of all classes it 
compiled itself and were loaded and offer a new ~ 
GroovyClassLoader#finalCleanup() method that removes meta information 
for all these classes so that they would become immediately eligible for 
garbage collection? (I guess InvokerHelper.removeClass(clazz) and 
Introspector.flushFromCaches(clazz), but whatever is needed...)

This would not help with Groovy classes that were precompiled and 
loaded, say, with an URLClassLoader, but would help with ones that were 
dynamically compiled at runtime.

Alain


On 17.05.16 00:35, John Wagenleitner wrote:
>
>
> On Mon, May 16, 2016 at 1:34 PM, Alain Stalder <astalder@span.ch 
> <mailto:astalder@span.ch>> wrote:
>
>     Thanks, I had not looked at the master branch, ClassInfo source
>     looks quite a bit cleaner there already :)
>
>     Regarding programmatic cleanup (GROOVY-7646), I think that is a
>     good idea, but in the details there might be some obstacles.
>
>
> I definitely agree, many obstacles usually present themselves once you 
> scratch the surface. :)
>
>
>
>     For example this sequence of calls to GroovyShell:
>
>     def shell = new GroovyShell()
>     def script1 = shell.parse("42")
>     assert script1.class.name <http://script1.class.name> == "Script1"
>     def script2 = shell.parse("new Script1().run()")
>     assert script2.run() == 42
>
>     def script3 = shell.parse("99", "Nintetynine")
>     assert script3.class.name <http://script3.class.name> == "Nintetynine"
>
>     def file = new File("Fiftyfive.groovy")
>     file.setText("55")
>     def script4 = shell.parse(file)
>     assert script4.class.name <http://script4.class.name> == "Fiftyfive"
>
>     So, classes accumulate (in the GroovyClassLoader) and can be
>     addressed by their names in subsequent scripts. (And for more
>     complex script expressions, more than one class might be the
>     result of compilation, e.g. with closures or inner classes, enums,
>     etc.)
>
>
>
> This passes with the changes in place for GROOVY-7646, though calls to 
> parse don't call the added clean-up code. It still passes if I change 
> parse to run.
>
>
>
>     I would estimate that in the case where a script is run with a
>     name given automatically by the GroovyShell ("Script1", "Script2",
>     ...) it would be OK to do the cleanup (and I guess using
>     GroovyShell that way might be a very common case?), but when it
>     comes to explicitly named scripts, doing so might change behaviour
>     of existing code.
>
>
>
> For quite some time GroovyShell#evaluate(Reader,String filename) was 
> doing this kind of cleanup [1].  Cleanup meaning removing the 
> metaClass, the ClassInfo from the cache and the Introspector 
> beanInfoCache.  As long as the ClassLoader and any of it's classes are 
> still referenced the Classes that result from parse/run calls would 
> still be available.  But you are right, there are so many ways the 
> shell can be used it is difficult to tell what it might break.
>
>
>     I just took a look at GroovyScriptEngine which also has run()
>     methods and, if I remember correctly, it recompiles all scripts if
>     one of them changes (to get dependencies right), so in principle
>     lots of classes to cleanup for each time this happens. But I am
>     not sure if that is possible there, because there is also a
>     createScript() method, so possibly still objects/classes that are
>     in use around.
>
>     (And I have also just started to think about Grengine in that
>     context, my open source library for using Groovy in a Java VM (and
>     which almost nobody uses ;), there it might be easier to build in
>     such automatic removal because the approach is more structured,
>     although a bit less dynamic.)
>
>     Hmn, would really be great if there was a way to achieve constant
>     garbage collection of Groovy classes.
>
>
>
> I take constant to mean not waiting until heap/metaspace is filled 
> before collection.  If so, from that I've seen that would still 
> require some user intervention 
> (java.beans.Introspector.flushFromCaches(clazz)) in order to clear the 
> Introspector cache which keeps a Soft Reference to main method of a 
> Script class which in turns references the Class.
>
>
>
>     Alain
>

Mime
View raw message