cocoon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michel Lehon" <Michel.Le...@Outwares.com>
Subject RE: JDK1.2 MemoryStore & SoftReferences... Follow up.
Date Mon, 03 Apr 2000 09:12:44 GMT
> From: Jason Reid [mailto:jreid@agency.com]
>
> > 2°) Use a level of indirection between the object stored in the
> cache and
> > the SoftReference.
> > The holder object sitting between the two would use the finalizer to get
> the
> > same result as in 1°)... This works better but it requires allocation of
> > memory in the finalizer, and we know the system is low in
> memory (that why
> > it tries to clear the Reference to the object). So once in a while
> > OutOfMemoryErrors are thrown, this means this approach is not useful.
>
> I'm stuck as to why this approach requires allocation of memory
> in the Proxy
> object's finalizer...
>
> 	class MemoryStoreGCProxy
> 	{
> 		MemoryStore mstore;
>
> 		public MemoryStoreGCProxy(MemoryStore mem)
> 		{
> 			mstore = mem;
> 		}
>
> 		protected void finalize()
> 		{
> 			mstore.releaseSomeHardReferences();
> 		}
> 	}
>
> and just keep the MemoryStoreGCProxy as a SoftReference.

Exactly what I tried... but there are a few problems:
In the finalizer you need to recreate the SoftReference (They can be
reused), and you need to create a new MemoryStoreGCProxy, because finalizer
are only run once (even if you resurrect the object). You need to recreate
those so the next time the gc runs out of mem it will call you again. You
could postpone that creation for later, but then in the mean time you are
vulnerable to OutOfMemoryError, and since your hook is not yet in place you
can't help the system.

But in fact, the memory allocation causing the problem is not the one done
in the finalizer. It is the done that causes the gc to cleanup the
SoftReference.

Since my last follow up... I tried Niclas Hedhman's ideas which where :
- Try again overriding methods of SoftReference (I had only tried clear,
without success).
- Try to implement a delegation algorithm (Just need to find a hook that
gets called when the System is running out of mem).

I tried overidding the 'clear()' and 'enqueue()' method of SoftReference,
but neither of the two is called by the GC. I also tried to look at a method
to override in the ReferenceQueue class, but there is none of use (kind of
funny but this queue has not public/protected enqueue method, only remove()
and poll()).

So it leaves the finalize() method of an object kept by a SoftReference,
that finalize method would do the cleanup (exactly what your code sample
does).

But in the mean time I came upon one more problem... finalizer are usually
too late.
Running tests with the -verbose:gc option, one can see this sequence:
- JVM runs out of memory.
- Tries to reclaim some by cleaning up garbage.
- Tires to cleanup SoftReferences if not enough memory could be reclaimed
and the Heap cannot be grown anymore.
- if there is still not enough memory available, it throws OutOfMemoryError.

In the process the 'cleaned' objects are put on a 'finalization queue' that
seems to run asynchronously.
So the JVM runs out of mem, and then the finalization frees some... but it
is too late.
So it really looks like a dead end (for the moment).

for those who want to take a look at the last test, the source is available
at
http://www.outwares.com/cocoon/SimpleTest3.java (Nice name ?).
Comments more than welcome.


Unless someone has more ideas to submit ?

>
> At this point, I'm not yet sure how the MemoryStore knows when to stop
> releasing HardReferences, though, or when to see to the construction of
> a new MemoryStoreGCProxy.

I'd like to be tackling that problem instead. I think Stephano's
implementation of MemoryStore shows how it can be done.

> So, I obviously haven't come up with the
> ultimate scheme for a Memory Store.  I was just trying to get your
> perspective on why the finalizer would need more memory.

Hope you see my point.

>
> Granted, the above doesn't solve the problem with HotSpot
> seemingly treating
> SoftReferences as WeakReferences, but if that's the case, then an
> optimal MemoryStore might be very difficult on such a buggy
> implementation.

Keep in mind that I tested only with HotSpot 2.0RC1 (this is not a final).

>
> 	>	Jason Reid
> 		Technical Consultant
> 		AGENCY.COM
> 		E jreid@agency.com
> 		http://www.agency.com
>

Michel Lehon
Michel.lehon@Outwares.com
SAS DataWarehousing & Web Enablement.


Mime
View raw message