cocoon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michel Lehon" <Michel.Le...@Outwares.com>
Subject JDK1.2 MemoryStore & SoftReferences... Follow up.
Date Thu, 23 Mar 2000 10:31:20 GMT
Hi everyone,

Sometime ago (almost 1 month), I posted a test implementation of a
SoftMemoryStore for Cocoon using JDK1.2 and SoftReferences (for those who
missed that mail but are curious it is dated 28 Feb 2000 and titled "Memory
Store and JDK 1.2").

Why would this be useful ? The main reason is to be nice with other
servlets. The current implementation of the MemoryStore removes objects from
the Store when Cocoon runs low in memory. This SoftMemoryStore would remove
objects from the cocoon cache, regardless of location, it would just clean
up when the system needs memory. This could prevent the Servlet engine from
crashing out of memory.

There was a few problems with that implementation...
Those problems are related to the SoftReferences.

I went hunting down those lines for sometime so here is some information I
gathered.

Just for the recap. the aim of this implementation was to better mamage the
Cache of Cocoon and make it grow as big as it can, but without throwing
OutOfMemoryErrors.
References are supposed to help you with that.

There are two kind of References to help us:

WeakReference: Used to keep a link to an object without preventing the
garbage collector from reclaiming it at ANY time. In JDK 1.2.2 (under Win98
& WinNT4WS, tested both with classic WM & HotSpot2RC1) testing shows that
WeakReferences are cleared as soon as possible (of course after all hard
refs are cleared).
This does not make them suitable for the MemoryStore as Stored objects would
be removed from the Store immediatly.

SoftReference: Same as WeakReference, except that they are supposed to be
reclaimed when the System memory runs low but before throwing an
OutOfMemoryException. They indeed behave like that. The problem is that all
SoftReferences are cleared in one go (under the Classic VM), which means
that when the system is running low in memory is empties the MemoryStore.
The real problem is that the specification does not require the GC to do
anything better, it can use a Least recently used scheme (The source code of
SoftReference even keeps a time stamp of the last access). In fact the
HotSpot2RC1 VM is even worse as it clears the SoftReferences before the
memory runs low (it looks like HotSpot treats SoftRefs and WeakRefs the
same).

I tried a few dirty tricks to see how to chose the SoftRefence to be
cleared... But without much success. Here is a desciption of some of the two
main tricks.

1°) Subclass SoftReference and overide the clear method... would allow to
refuse to clear if this object is not the worst in the Store. Nice try but
the clear method is not called by the GC, it goes directly inside the object
and clears the private instance variable.

2°) Use a level of indirection between the object stored in the cache and
the SoftReference.
The holder object sitting between the two would use the finalizer to get the
same result as in 1°)... This works better but it requires allocation of
memory in the finalizer, and we know the system is low in memory (that why
it tries to clear the Reference to the object). So once in a while
OutOfMemoryErrors are thrown, this means this approach is not useful.

So to conclude I did not succeed (yet) in imporving the SoftMemoryStore.
The current implementation is still the best I can do for the momement, I'll
carry on looking but If anyone has any information on how to improve it...
It would be welcome.

Sorry for the long mail... but I think it was more than time for a follow
up.

Michel Lehon
Michel.Lehon@Outwares.com
SAS DataWarehousing & Web Enablement.


Mime
View raw message