groovy-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alain Stalder <astal...@span.ch>
Subject Re: Improve Groovy class loading performance and memory management
Date Mon, 16 May 2016 10:46:07 GMT
In order to get a better understanding, I made two configurable changes 
in ClassInfo, in a branch from the GROOVY_2_4_6 tag (ClassInfo is still 
practically the same in the GROOVY_2_4_X branch):

- -Dgctest.classreftype=(hard|soft|weak|phantom), where hard=as today, 
soft=SoftReference
- -Dgctest.cacheclassvalue=(true|false), if true and using ClassValue, 
then do not cache it

See here:
https://github.com/jexler/groovy/compare/GROOVY_2_4_6...jexler:f92c2866653208ad68db5580b5bf9febc347fe1d

Compiled Groovy JAR:
https://www.jexler.net/groovy-2.4.6-gctest.jar

First thing I learned was that you cannot get the value of a 
PhantomReference, it always returns null, by design. From its Javadoc: 
"In order to ensure that a reclaimable object remains so, the referent 
of a phantom reference may not be retrieved: The get method of a phantom 
reference always returns null."

(By the way, this very probably means that the already existing 
PhantomReference myThread in ClassInfo makes no sense.)

Then I ran a full matrix of tests:

------------------------------------------------------------------------
  same loader | use class value | cache class value | hard | soft | weak
------------------------------------------------------------------------
  YES         | YES             | YES               | FAIL | FAIL | FAIL
  YES         | YES             | NO                | FAIL | FAIL | FAIL
  YES         | NO              | --                |  OK  |  OK* | OK*
------------------------------------------------------------------------
  NO          | YES             | YES               |  OK  |  OK* | OK*
  NO          | YES             | NO                |  OK  |  OK* | OK*
  NO          | NO              | --                | FAIL |  OK* | OK*
------------------------------------------------------------------------

- "same loader" <=> java [opts] -XX:MaxMetaspaceSize=64m -Xmx512m -cp . 
ClassGCTester -cp groovy-2.4.6-gctest.jar:filling/ -parent null -classes 
GroovyFilling
- not "same loader" <=> java [opts] -XX:MaxMetaspaceSize=64m -Xmx512m 
-cp .:groovy-2.4.6-gctest.jar ClassGCTester -cp filling/ -parent tester 
-classes GroovyFilling
- "use class value" <=> -Dgroovy.use.classvalue=<true|false>
- "cache class value" <=> -Dgctest.cacheclassvalue=<true|false>
- "hard"|"soft"|"weak" <=> -Dgctest.classreftype=<hard|soft|weak>

* Garbage collection in all cases still only when the limit on Metaspace 
or Heap is reached.

So:
- Caching ClassValue or not made no difference.
- Using weak oder soft references did not help when using ClassValue.
- When not using ClassValue, using weak or soft references helped. :)

Actually the latter is also reflected (as I noticed in retrospect) by 
the pull request by John Wagenleitner for "GROOVY-7683 - Memory leak 
when using Groovy as JSR-223 scripting language": 
https://github.com/apache/groovy/pull/219/files

There a WeakReference is used.

Which brings my mind back to my question regarding whether it is "good 
architecture" to have a reference to the class in ClassInfo (or any 
other metadata associated with a class) - again, I mean fundamentally, 
independently of whether this is an option for a Groovy 2.4.7 or even 
anything before a Groovy 3, because I fear it would likely require to 
change several Groovy APIs and internals.

If using now a WeakReference or SoftReferencefor the class reference in 
ClassInfo instead of a hard reference, you now have to handle the case 
where the class is already null because it has been garbage collected. 
(Actually this is in principle more likely with a WeakReference than 
with a SoftReference, so I would rather tend to favor SoftReference 
because class GC so far only kicks in when a memory limit is reached 
anyway, but likely it makes no difference in practice exactly for the 
same reason. Actually, this may even save the situation, maybe in 
practice you never get the Reference to return null because classes and 
ClassInfo are only garbage collected together when the memory limit is 
reached in a Java VM that does nothing else then, but I am not sure...)

My argument is still the same: ClassInfo (or other assiociated metadata) 
only makes sense if you have your hands on a class (or an instance of 
it) to apply it to. The one who wants to do something with the 
class/instance has it and in principle can pass it down to ClassInfo in 
order to extract whatever is needed. If there is no "client" with a 
class/instance, there is no need to create ClassInfo (or similar). And 
if the class is garbage collected, automatically ClassInfo cannot be 
accessed with such queries any more, and then also the JVM bug with 
ClassValue would no longer affect Groovy, ClassValue could be used again 
by default.

But I don't want to make too much of this.

Using a WeakReference or SoftReference for the class reference in 
ClassInfo would already be step forward, at least no better realistic 
ideas from my side at the moment...

Alain

On 15.05.16 12:37, Jochen Theodorou wrote:
> On 15.05.2016 10:39, Alain Stalder wrote:
>> Thanks, that clarifies a lot to me, especially SoftReference.
>>
>> So with Groovy it is only realistic to have GC of classes (and attached
>> ClassInfo) kick in once a limit on Metaspace/PermGen (or Heap) is
>> reached - fine with me, no point to try to "outrun the bear"... :)
>
> well... I do think the ClassValue version should not have this 
> behaviour. But for this I think we would have to ensure not to keep 
> any references to the ClassValue anywhere in a global strucutre. Not 
> even as a WeakReference... PhantomReference would probably be ok... 
> but I find the usages for PhantomReferences quite rare...and not 
> fitting here I guess
>
>> A general question (current implementation and most likely APIs to keep
>> aside): Why does ClassInfo need a reference to the class? To me the use
>> case would be that you have an Groovy object or a Groovy class and want
>> to do something with it (call a static or instance method, for example),
>> so you only need to find ClassInfo from the class and then maybe pass
>> the class temporarily just for doing things, but don't need it a
>> reference back from ClassInfo.
>
> ClassInfo represents a cached reflective information of a Class, plus 
> some more internal stuff. To create that structure you need the Class. 
> And if you do not want to do it eager, you need to keep a reference... 
> at least till after init. Of course that does not have to be a 
> SoftReference.
>
> [...]
>> This allows, for example, two produce two of the known
>> "OutOfMemoryError: Metaspace|PermGen" issues with Groovy 2.4.6, as 
>> follows.
> [...]
>
> good job
>
> bye Jochen
>
>


Mime
View raw message