cocoon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ard Schrijvers" <a.schrijv...@hippo.nl>
Subject RE: Continuations consume ram, possible solutions?
Date Mon, 31 Jul 2006 10:15:57 GMT
Hello,

> 
> +1, much simpler to implement, much simpler to use, no hidden 
> side effects.
> 
> Ard, you carrying this out? :) :)
> 

Yes, I will try, but I have some doubts/questions/ideas that I would like to share and find
the best solution to implement this new janitor:

1) I will try to implement it initially for the cocoon-2.1.X version.
2) In my opinion, it doesn't make sense to see the excalibur StoreJanitor and the "ContinuationJanitor"
as two seperate janitors trying to free memory when the JVM is low on memory. The StoreJanitor
and the ContinuationJanitor are in my opinion one and the same, because I think depending
on some choices, the janitor should either try to free memory from cache, or from the continuations
(perhaps in a special case, from both.)

I have been thinking about changing the StoreJanitor anyway, to be able to have stores defined
that are never tried to be cleared (like the defaultTransientStore containing for example
30 compiled xsl's. I never want the StoreJanitor removing these)

As we will also have the continuations to be managed, the name StoreJanitor seems inappropriate
to me. Perhaps just CocoonJanitor? 

Anyway, stored are registered to the "CocoonJanitor" (call it from now on CocoonJanitor) already.
I am not sure wether this is possible for continuations, but otherwise I would have to do
a manager.hasService(ContinuationsManager.role), in the CocoonJanitorImpl, which is a bad
solution, right? Does anybody have an idea how to register the ContinuationManager to the
CocoonJanitor.

3) When the maxcontinuations for continuations is reached, we have to options: 
	* hardlimit: adding one to the ContinuationManagerImpl means at the same moment actively
removing one from it according LRU. 
	* softlimit: betweem concurrent CocoonJanitor runs, the continuations are allowed to exceed
the maximum. When this maximum is exceeded, the CocoonJanitor will prune it back to maxcontinuations
according LRU.

I am in favor of the softlimit, to keep insertion of new continuations as lean and fast as
possible. WDYT? 

4) When the CocoonJanitor runs, and JVM turns out to be low on memory, we have multiple choices
	* Try to free from caches which are configured to be "freeable" (a nicer word I hope to find
for this). I also want the StoreJanitor to free cache of all configured caches which are "freeable"
at once, and do not do it like it does now: free from one cache, and jump to the next one
in the next round (10 sec for example later)
	* Try to free from continuations
	* Try to free from both.

We have to find some strategy on this one, but I think it might result in a quite heuristic
solution: For example:

1) If maxcontinuations is exceeded, try freeing memory from continuations
2) If both maxobjects (for all freeable caches) AND maxcontinuations is not reached, where
should we free from? Free from both? This does not always make sense. Free according to the
largest occupied percentage? I am not sure about this, anybody? 
3) when (2), should we try freeing from continuation or cache according absolute numbers (which
has the largest items in use) or relative? Should we make it configurable? This will be very
hard for common users to grasp or tune. Of course, well documented configuration might largely
help. But it remains quite heuristi. 

I think eviction policiy according LRU is very easy, because already the continuations are
present sorted by expiring time (which gets increased when it is used again (I think at least
it works like this)).

WDYT?

Regards Ard

> Simone
> 
> Torsten Curdt wrote:
> 
> > <snip/>
> >
> > IMO the only way to solve this transparently is to more accressively
> > expire and limit the number of continuations. It would make sense to
> > come up with a LRU list of continuations per session. This 
> list has a
> > maximum size that can defined. So the required maximum can is
> > predictable. Generating more continuations means using free slots or
> > throwing away the oldest ones in that LRU list. The janitor would
> > basically only go through the list and expire to free the slots in
> > that list.
> >
> > cheers
> > -- 
> > Torsten
> 
> 

Mime
View raw message