Mailing-List: contact dev-help@groovy.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@groovy.apache.org
Subject: Re: Improve Groovy class loading performance and memory management
To: dev@groovy.apache.org
References: <57351E2A.2070304@span.ch> <573835A4.2060301@span.ch>
 <5738517B.5010301@gmx.org> <5739A4EF.1020002@span.ch>
From: Jochen Theodorou <blackdrag@gmx.org>
Message-ID: <5739B695.5070408@gmx.org>
Date: Mon, 16 May 2016 14:01:25 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101
 Thunderbird/38.4.0
MIME-Version: 1.0
In-Reply-To: <5739A4EF.1020002@span.ch>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
archived-at: Mon, 16 May 2016 12:01:34 -0000

On 16.05.2016 12:46, Alain Stalder wrote:
> In order to get a better understanding, I made two configurable changes
> in ClassInfo, in a branch from the GROOVY_2_4_6 tag (ClassInfo is still
> practically the same in the GROOVY_2_4_X branch):
>
> - -Dgctest.classreftype=(hard|soft|weak|phantom), where hard=as today,
> soft=SoftReference
> - -Dgctest.cacheclassvalue=(true|false), if true and using ClassValue,
> then do not cache it
>
> See here:
> https://github.com/jexler/groovy/compare/GROOVY_2_4_6...jexler:f92c2866653208ad68db5580b5bf9febc347fe1d
>
>
> Compiled Groovy JAR:
> https://www.jexler.net/groovy-2.4.6-gctest.jar
[...]
> Then I ran a full matrix of tests:
>
> ------------------------------------------------------------------------
>   same loader | use class value | cache class value | hard | soft | weak
> ------------------------------------------------------------------------
>   YES         | YES             | YES               | FAIL | FAIL | FAIL
>   YES         | YES             | NO                | FAIL | FAIL | FAIL
>   YES         | NO              | --                |  OK  |  OK* | OK*
> ------------------------------------------------------------------------
>   NO          | YES             | YES               |  OK  |  OK* | OK*
>   NO          | YES             | NO                |  OK  |  OK* | OK*
>   NO          | NO              | --                | FAIL |  OK* | OK*
> ------------------------------------------------------------------------
>
> - "same loader" <=> java [opts] -XX:MaxMetaspaceSize=64m -Xmx512m -cp .
> ClassGCTester -cp groovy-2.4.6-gctest.jar:filling/ -parent null -classes
> GroovyFilling
> - not "same loader" <=> java [opts] -XX:MaxMetaspaceSize=64m -Xmx512m
> -cp .:groovy-2.4.6-gctest.jar ClassGCTester -cp filling/ -parent tester
> -classes GroovyFilling
> - "use class value" <=> -Dgroovy.use.classvalue=<true|false>
> - "cache class value" <=> -Dgctest.cacheclassvalue=<true|false>
> - "hard"|"soft"|"weak" <=> -Dgctest.classreftype=<hard|soft|weak>
>
> * Garbage collection in all cases still only when the limit on Metaspace
> or Heap is reached.
>
> So:
> - Caching ClassValue or not made no difference.
> - Using weak oder soft references did not help when using ClassValue.
> - When not using ClassValue, using weak or soft references helped. :)

Even if hard references had been used everywhere, it is only for a 
single iteration. It means unless data leaks into a more global 
structure, it must be collectable. So the non-ClassValue version working 
in this scenario is no sign of correctness for the memory awareness of 
the used structures at all.


> Actually the latter is also reflected (as I noticed in retrospect) by
> the pull request by John Wagenleitner for "GROOVY-7683 - Memory leak
> when using Groovy as JSR-223 scripting language":
> https://github.com/apache/groovy/pull/219/files
>
> There a WeakReference is used.

yes, I think that is something we should do.

> Which brings my mind back to my question regarding whether it is "good
> architecture" to have a reference to the class in ClassInfo (or any
> other metadata associated with a class) - again, I mean fundamentally,
> independently of whether this is an option for a Groovy 2.4.7 or even
> anything before a Groovy 3, because I fear it would likely require to
> change several Groovy APIs and internals.

ok, let´s assume the ClassInfo does not reference the class, then as 
soon as you have a MetaClass, you have a reference to the class again. 
If not there, then in the method accessors...

bye Jochen