cocoon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sylvain Wallez <sylvain.wal...@anyware-tech.com>
Subject Re: Store Janitor Hangs System
Date Mon, 28 Jan 2002 12:43:04 GMT
Stefano Mazzocchi wrote:

<snip/>

>
>Be careful when going down this road: the JLS doesn't say anything about
>'how' the weak references should be cleaned. And Java 1.3 implements it
>in the most memory efficient possible way: if the GC gets a weak
>reference, all of them are removed.
>
>Now, imagine the case: you have spent a couple of hours building up and
>tuning your cache using highly complex statistical analysis... but you
>saved them into a store that is implemented using weak references.
>
>You hit the heap ceiling and the GC kicks in.
>
>If finds one of those weak references, but it doesn't have the ability
>to ask you (well, the store) about 'which is weaker', so, wooosh,
>everything is gone.
>
>Sure, you probably collect 40Mb or heap, but then your perceived
>performance drops and your cache system works in a 'saw-like' figure
>
>
>cache size
> ^ 
> |     /|   /|
> |    / |  / |
> |   /  | /  |
> |  /   |/   |/
> +-----------------> time
>        ^
>    GC kicks in 
>
>and since the cache size is linearly proportional with the perceived
>efficiency of the cache, Cocoon will appear to be 'fast at times' (to be
>kind) or 'awefully bumping' (to be more reasonable).
>
>The solution I thought for this problem is simple but I don't know if it
>works (never tested it real life):
>
> 1) partition the memory store into 'clusters' which are connected using
>hard references.
>
> 2) these 'clusters' are the real containers of the stored entities and
>such entities are weak-referenced from the 'clusters'
>
>The idea is to have something like this:
>
> store +==> cluster +---> entity
>       |            +---> entity
>       |            +---> entity
>       |
>       +==> cluster +---> entity
>                    +---> entity
>                    +---> entity
>
>where:
> 
> ===> hard reference
> ---> soft reference
>
Are you meaning "soft" or "weak" reference ?

Soft references (java.lang.ref.SoftReference) should be good to look at, 
since the GC is smarter with them : only the least recently used 
references are cleared. The javadoc for SoftReference even gives some 
hints to implement caches.

Also, here are two cache systems I know of (without having studied 
them). Maybe they're worth looking at :
- http://jpolycache.sourceforge.net/
- http://www.opensymphony.com/oscache/

>
>
>and the key concept is that, whenever the GC needs to free some memory
>that is occupied by weak references, the avalanche cleanup will be
>'stopped' at a cluster level, saving the other clusters.
>
>Now, I'm not sure this works, but if it does, we have a reasonable
>solution to the problem.
>
>>Sorry for all that crap with Store and everything.
>>
>Don't worry, it takes time to do it right but I'm sure we'll get there.
>
>Probably we need to discuss its design *before* implementing it :)
>
Design before coding is always a good thing ;)

Sylvain

-- 
Sylvain Wallez
Anyware Technologies - http://www.anyware-tech.com




---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org


Mime
View raw message