cocoon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gerhard Froehlich" <g-froehl...@gmx.de>
Subject RE: Store Janitor Hangs System
Date Sun, 27 Jan 2002 11:58:37 GMT
Hi,

>From: Peter Hargreaves [mailto:pdh@totalise.co.uk]
>
>Hi Gerhard and Team,
>
>I'm posting in this group now I think I found a design problem with 
>store-janitor.

Slowly but constant this Store issues are a PITA for me :-/. I don't know
why this is in Java that difficult to control a little bit memory.
That's the price for automatic Memory Management!
(Nothing against you Peter!)

>I think store-janitor works fine when it can free sufficient memory by 
>calling the finalize and garbage collection. But if it needs to free the 
>store it locks up!
>
>The reason it appears to lock up (I think), is because it calls the 
>garbage collector too often - i.e. every time it frees an item from a 
>store. Once called the store-janitor will free an item and call the 
>garbage collector, repeating both until it has freed sufficient memory. 
>If one item frees 4.44k and the garbage collector takes 2844ms then the 
>time taken to free 20m will be 3.5 hours - during which time the system 
>appears to be locked!
>
>The following is a brief summary of my analysis:
>
>With the following settings:
>Xms=0m
>Xmx=140m
>heapsize =100000000
>freememory=20000000
>
>Using Optimizeit I watched my java heap 'total' and 'in-use' growing. 
>When total heapsize gets to 100m the store-janitor kicks in and frees a 
>chunk of memory. This repeats every ten seconds but each time it frees 
>less memory until it is no longer able to free memory and locks up. When 
>locked up the garbage collector is running about 99% of the time.
>
>Looking at the store-janitor code and my debug log I can see exactly 
>what happens when it locks.
>
>I notice, examining Store-janitorImpl.run() that, if the garbage 
>collector alone manages to clear sufficient memory, no store items are 
>cleared (as expected). This is possibly all that is happening when 
>store-janitor appears to work! However if that is not sufficient the 
>following loop is started.
>
>                 synchronized (this) {
>                     while (this.memoryLow() && 
>this.getStoreList().size() > 0) {
>                         this.freeMemory();
>                     }
>                     this.resetIndex();
>                 }
>
>this.memoryLow() tests to see if sufficient is free and prints Debug info.
>this.freeMemory() removes an item, runs garbage collection, and prints 
>Debug info.
>
> From my log files I can see this loop being executed. Each time round 
>the loop I can see how much memory is freed and how long it takes. 
>Typically it can take seconds to free one item which can be only a few 
>kb. So, in my case with 'freememory' set to 20m, it is likely to take a 
>few hours to free sufficient memory!!
>
>I've noticed when running Optimizeit that although garbage collection 
>can be quick, it sometimes takes a few seconds. If it has just been run, 
>and you force it to run again, then it seems to take longer. If it can't 
>find much to freeup, it seems to do a more time consuming search for 
>garbage.

This all makes sense too me.
Then we need a limit how often the GC can be called. Maybe 100 loops
should be enough. After that it waits as configured (threadinterval)
before the next execution.

Maybe there is a more elegent solution.

I actually develop in jakarta-commons a store (again) with Weak References. 
This is a much better solution. No Threads, no GC calls. When it is ready
to introduce I'll explain you!

Sorry for all that crap with Store and everything.

  Gerhard

--------------------------
Hey! It compiles! Ship it!
--------------------------


---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org


Mime
View raw message