groovy-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alain Stalder <astal...@span.ch>
Subject Re: Improve Groovy class loading performance and memory management
Date Wed, 18 May 2016 09:59:15 GMT
Looking at that code for GlobalClassset below now, the itemsMap is only 
used for two things:
- put(), where performance is not crucial because it happens only once 
per loaded class (which is relatively expensive anyway)
- get(), where performance is crucial
but no iterations or removals etc. necessary.

Could it work to try get() first without synchronize and if it returns 
null or throws an exception, just try again in a synchronize(itemsMap) 
block? I have no experience with doing such a thing with a 
(Weak)HashMap... Could a synchronized put() fail if there is an 
unsycnchronized get() at the same time?

I think I will try that unless someone tells me it can't work...?

If it was worth a try:

Any tips on tests I could run to compare performance of Groovy master 
with this branch (and verify that it is thread-safe)?
I know there are tests in the benchmark directory of the groovy sources 
- which one(s) could I maybe run for this?

Actually, the fact that classes would be collected on-the-fly would 
reduce the size of itemsMapm which would in principle shorten access 
times, but maybe not significantly - would have to be seen then...

Alain


On 18.05.16 10:02, Alain Stalder wrote:
>
> On 18.05.16 09:10, Jochen Theodorou wrote:
>>>      private static class GlobalClassSet {
>>>
>>>          //private final ManagedLinkedList<ClassInfo> items = new
>>> ManagedLinkedList<ClassInfo>(weakBundle);
>>>          private final WeakHashMap<Class,WeakReference<ClassInfo>>
items 
>>>
>>> = new WeakHashMap<Class,WeakReference<ClassInfo>>();
>>
>> would be actually interesting to keep the list and see if it can 
>> still garbage collect
>>
> Looks like it can. (As I would have expected because 
> ClassInfo.remove(clazz) did not touch that list before and that was 
> sufficient to get GC on-the-fly provided you also do 
> Introspector.flushFromCaches(clazz) ):
>
> --
>     private static class GlobalClassSet {
>
>         private final ManagedLinkedList<ClassInfo> itemsList = new 
> ManagedLinkedList<ClassInfo>(weakBundle);
>         private final WeakHashMap<Class,WeakReference<ClassInfo>> 
> itemsMap = new WeakHashMap<Class,WeakReference<ClassInfo>>();
>
>         public int size(){
>             return values().size();
>         }
>
>         public int fullSize(){
>             return values().size();
>         }
>
>         public Collection<ClassInfo> values(){
>             synchronized(itemsList){
>                 return Arrays.asList(itemsList.toArray(new ClassInfo[0]));
>             }
>         }
>
>         public void add(ClassInfo value){
>             synchronized(itemsList) {
>                 itemsList.add(value);
>             }
>             synchronized(itemsMap) {
>                 itemsMap.put(value.klazz, new 
> WeakReference<ClassInfo>(value));
>             }
>         }
>
>         public ClassInfo get(Class cls) {
>             WeakReference<ClassInfo> ref;
>             synchronized(itemsMap) {
>                 ref = itemsMap.get(cls);
>             }
>             ClassInfo info;
>             if (ref == null) {
>                 //System.out.println("ClassInfo Ref is null: " + 
> cls.getName());
>                 info = new ClassInfo(cls);
>                 synchronized (itemsMap) {
>                     itemsMap.put(cls, new WeakReference<ClassInfo>(info));
>                 }
>                 return info;
>             }
>             info = ref.get();
>             if (info == null) {
>                 //System.out.println("ClassInfo is null: " + 
> cls.getName());
>                 info = new ClassInfo(cls);
>                 itemsMap.put(cls, new WeakReference<ClassInfo>(info));
>                 return info;
>             }
>             return info;
>         }
>
>     }
> --
>
> $ java -XX:MaxMetaspaceSize=64m -Xmx512m -cp 
> .:groovy-2.5.0-SNAPSHOT.jar ClassGCTester -cp filling/ -parent tester 
> -classes GroovyFilling
> (does a Introspector.flushFromCaches(clazz) for each loaded class)
>
> Secs Test classes              Metaspace/PermGen Heap   Load time 
> Create time    Run time Cleanup time
>        #loaded  #remaining        used committed       used 
> committed     average     average     average      average
>    0         1           1       6.3m       6.5m      13.4m 245.5m     
> 0.890ms    14.308ms  0.026168ms   0.019285ms
>    1       435         435       8.9m      10.1m      22.1m 245.5m     
> 0.365ms     1.825ms  0.000064ms   0.000009ms
>    2      1202        1202      11.9m      14.6m      66.1m 245.5m     
> 0.280ms     1.314ms  0.000024ms   0.000001ms
>    3      2197        2197      15.7m      20.4m      83.8m 309.5m     
> 0.240ms     1.070ms  0.000010ms   0.000001ms
>    4      3247         966      11.0m      16.8m      16.5m 242.0m     
> 0.226ms     0.959ms  0.000006ms   0.000000ms
>    5      4396        2115      15.4m      20.3m      44.5m 238.0m     
> 0.208ms     0.886ms  0.000005ms   0.000000ms
>    6      5415        3134      19.3m      26.0m      54.4m 235.5m     
> 0.202ms     0.863ms  0.000009ms   0.000000ms
>    7      6458         667       9.8m      18.0m      94.7m 266.5m     
> 0.203ms     0.839ms  0.000003ms   0.000000ms
>    8      7550        1759      14.0m      21.4m     122.0m 268.5m     
> 0.198ms     0.821ms  0.000003ms   0.000000ms
>    9      8748        2957      18.6m      25.9m      46.3m 268.5m     
> 0.191ms     0.799ms  0.000003ms   0.000000ms
> [...]
>
> Very interesting because the list contains references to the class and 
> yet it can be garbage collected on-the-fly... Maybe that could help to 
> find a solution?
>
> Alain
>


Mime
View raw message