cocoon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gerhard Froehlich" <>
Subject RE: Store Janitor Hangs System
Date Sun, 27 Jan 2002 11:58:37 GMT

>From: Peter Hargreaves []
>Hi Gerhard and Team,
>I'm posting in this group now I think I found a design problem with 

Slowly but constant this Store issues are a PITA for me :-/. I don't know
why this is in Java that difficult to control a little bit memory.
That's the price for automatic Memory Management!
(Nothing against you Peter!)

>I think store-janitor works fine when it can free sufficient memory by 
>calling the finalize and garbage collection. But if it needs to free the 
>store it locks up!
>The reason it appears to lock up (I think), is because it calls the 
>garbage collector too often - i.e. every time it frees an item from a 
>store. Once called the store-janitor will free an item and call the 
>garbage collector, repeating both until it has freed sufficient memory. 
>If one item frees 4.44k and the garbage collector takes 2844ms then the 
>time taken to free 20m will be 3.5 hours - during which time the system 
>appears to be locked!
>The following is a brief summary of my analysis:
>With the following settings:
>heapsize =100000000
>Using Optimizeit I watched my java heap 'total' and 'in-use' growing. 
>When total heapsize gets to 100m the store-janitor kicks in and frees a 
>chunk of memory. This repeats every ten seconds but each time it frees 
>less memory until it is no longer able to free memory and locks up. When 
>locked up the garbage collector is running about 99% of the time.
>Looking at the store-janitor code and my debug log I can see exactly 
>what happens when it locks.
>I notice, examining that, if the garbage 
>collector alone manages to clear sufficient memory, no store items are 
>cleared (as expected). This is possibly all that is happening when 
>store-janitor appears to work! However if that is not sufficient the 
>following loop is started.
>                 synchronized (this) {
>                     while (this.memoryLow() && 
>this.getStoreList().size() > 0) {
>                         this.freeMemory();
>                     }
>                     this.resetIndex();
>                 }
>this.memoryLow() tests to see if sufficient is free and prints Debug info.
>this.freeMemory() removes an item, runs garbage collection, and prints 
>Debug info.
> From my log files I can see this loop being executed. Each time round 
>the loop I can see how much memory is freed and how long it takes. 
>Typically it can take seconds to free one item which can be only a few 
>kb. So, in my case with 'freememory' set to 20m, it is likely to take a 
>few hours to free sufficient memory!!
>I've noticed when running Optimizeit that although garbage collection 
>can be quick, it sometimes takes a few seconds. If it has just been run, 
>and you force it to run again, then it seems to take longer. If it can't 
>find much to freeup, it seems to do a more time consuming search for 

This all makes sense too me.
Then we need a limit how often the GC can be called. Maybe 100 loops
should be enough. After that it waits as configured (threadinterval)
before the next execution.

Maybe there is a more elegent solution.

I actually develop in jakarta-commons a store (again) with Weak References. 
This is a much better solution. No Threads, no GC calls. When it is ready
to introduce I'll explain you!

Sorry for all that crap with Store and everything.


Hey! It compiles! Ship it!

To unsubscribe, e-mail:
For additional commands, email:

View raw message