Mailing-List: contact river-dev-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: river-dev@incubator.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
Message-ID: <4BC3931A.1070800@zeus.net.au>
Date: Tue, 13 Apr 2010 07:39:38 +1000
From: Peter Firmstone <jini@zeus.net.au>
User-Agent: Thunderbird 2.0.0.14 (X11/20080531)
MIME-Version: 1.0
To: river-dev@incubator.apache.org
Subject: Re: ClassLoader and Class Garbage Collection issues with
 Serialization.
References: <4BC124EA.1070504@zeus.net.au>
 <B1CF6B3997B0D4499EF6043F5388D6B8055B18BE@TEW-MAIL.global.avidww.com>
 <4BC3371D.1000908@wonderly.org>
In-Reply-To: <4BC3371D.1000908@wonderly.org>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Awesome thanks, I found the bug database record.

http://bugs.sun.com/view_bug.do?bug_id=6232010

I'll have to see if I can track down the CDC source code and see if it 
too has the fix.

For what I'd like to do with ClassLoaders, this bug would have been a 
show stopper.

You can just imagine my thoughts, stumbling across it, while theorising 
about how to reduce duplication of class files, by sharing code between 
services for services utilising common packages.  I'd also like to 
return results from lookup as a stream, ordered by common bytecode, so 
the local jvm garbage collects unwanted results during stream 
inspection, to avoid a memory explosion.

Cheers,

Peter.

Gregg Wonderly wrote:
> It was my understanding that this issue had been solved as well.
>
> Gregg Wonderly
>
> Christopher Dolan wrote:
>> This is news to me.  Looking at the JDK 1.5 source code, I can see that
>> ObjectStreamClass$Caches uses weak references for the classes.  So
>> either this is a solved problem as of 1.5, or there's something
>> non-obvious in the implementation that I've overlooked.
>>
>> Chris
>>
>> -----Original Message-----
>> From: Peter Firmstone [mailto:jini@zeus.net.au] Sent: Saturday, April 
>> 10, 2010 8:25 PM
>> To: river-dev@incubator.apache.org
>> Subject: ClassLoader and Class Garbage Collection issues with
>> Serialization.
>>
>> I have stumbled across a troubling problem with Serialization 
>> relating to Garbage Collection of Classes and ClassLoaders and was 
>> hoping someone
>>
>> might be able to shed some light on the issue.
>>
>> Is it really true that the more objects you distribute, the greater your
>>
>> memory consumption because Class files and ClassLoaders cannot be 
>> garbage collected?
>>
>> Regards,
>>
>> Peter.
>>
>> The issue can be found here:
>> http://www.ibm.com/developerworks/java/library/j-dclp3/index.html
>>
>> And here's the relevant information, pasted from the link:
>>
>>
>>   Problems related to garbage collection and serialization
>>
>> The garbage collector interacts closely with the class loader. Among 
>> other things, the collector examines the class loader data structures to
>>
>> determine which classes are /live/ -- that is, are not garbage 
>> collectable. This can often lead to some unexpected problems.
>>
>> Figure 2 illustrates a situation where serialization affects the garbage
>>
>> collection (GC) of classes and a class loader in an unexpected way:
>>
>>
>>     *Figure 2. Serialization example*
>>
>> Serialization example
>>
>> In this example, |SerializationTest| instantiates a |URLClassLoader|, 
>> called |loader|. After loading |SerializationClass|, the class loader is
>>
>> dereferenced. The expectation is that this will allow the classes loaded
>>
>> by it to be garbage collected. The code for these classes is illustrated
>>
>> in Listings 9 and 10:
>>
>>
>> *Listing 9. SerializationTest.java*
>>
>> import java.net.MalformedURLException;
>> import java.net.URL;
>> import java.net.URLClassLoader;
>>
>> public class SerializationTest extends ClassLoader {
>>
>>    public static void main(String args[]) {
>>       try {
>>          URLClassLoader loader = new URLClassLoader(new URL[] { new URL(
>>                "file://C:/CL_Article/Serialization/dir1/") });
>>          System.out.println("Loading SerializationClass");
>>          Class c = loader.loadClass("SerializationClass");
>>          System.out.println("Creating an instance of
>> SerializationClass");
>>          c.newInstance();
>>          System.out.println("Dereferencing the class loader");
>>          c = null;
>>          loader = null;
>>                   System.out.println("Running GC...");
>>          System.gc();
>>          System.out.println("Triggering a Javadump");
>>          com.ibm.jvm.Dump.JavaDump();
>>                } catch (MalformedURLException e) {
>>          e.printStackTrace();
>>       } catch (InstantiationException e) {
>>          e.printStackTrace();
>>       } catch (IllegalAccessException e) {
>>          e.printStackTrace();
>>       } catch (ClassNotFoundException e) {
>>          e.printStackTrace();
>>       }
>>    }
>> }
>>
>>
>>
>> *Listing 10. SerializationClass.java*
>>
>> import java.io.File;
>> import java.io.FileOutputStream;
>> import java.io.ObjectOutputStream;
>> import java.io.Serializable;
>>
>> public class SerializationClass implements Serializable {
>>
>>     private static final long serialVersionUID = 5024741671582526226L;
>>
>>     public SerializationClass() {
>>         try {
>>             File file = new
>> File("C:/CL_Article/Serialization/test.txt");
>>             FileOutputStream fos = new FileOutputStream(file);
>>             ObjectOutputStream oos = new ObjectOutputStream(fos);
>>             oos.writeObject(this);
>>             oos.reset();
>>             oos.close();
>>             fos.close();
>>             oos = null;
>>             fos = null;
>>             file = null;
>>         } catch (Exception e) {
>>             e.printStackTrace();
>>         }
>>     }
>> }
>>
>>
>> Using a Javadump, it is possible to discover whether the class loader 
>> has been garbage collected. (See the first article in this series for 
>> more on using Javadump.) If the following section appears in the list of
>>
>> class loaders, then it has not been collected:
>>
>> ------a- Loader java/net/URLClassLoader(0x44DC6DE0), Shadow 0x00ADB6D8,
>>         Parent sun/misc/Launcher$AppClassLoader(0x00ADB7B0)         
>> Number of loaded classes 1         Number of cached classes 11      
>>         Allocation used for loaded classes 1              Package 
>> owner 0x00ADB6D8
>>      
>>
>> Though dereferencing a user-defined class loader seems like a way to 
>> ensure that the classes are garbage collected, this is not actually 
>> the case. In the previous example, the problem stems from the use of 
>> |java.io.ObjectOutputStream.writeObject(Object obj)| and its 
>> implications on GC.
>>
>> When |writeObject()| is invoked (to serialize |SerializationClass|), 
>> a reference to this class object is passed internally to 
>> |ObjectStreamClass| and stored in a lookup table (that is, in an 
>> internal cache). This reference is kept to speed up future serialization
>>
>> of the same class.
>>
>> When the class loader is dereferenced, the classes that it loaded are 
>> not garbage collectable. This is because there is a live reference to 
>> the |SerializationClass| class from the |ObjectStreamClass| lookup 
>> table. |ObjectStreamClass| is a primordial class and therefore is 
>> never garbage collected. The lookup table is referenced from a static 
>> field in
>>
>> |ObjectStreamClass| and is kept in the class itself rather than in an 
>> instance of it. As a result, the reference to |SerializationClass| 
>> exists for the lifetime of the JVM, and the class thus cannot be garbage
>>
>> collected. Importantly, the |SerializationClass| class has a 
>> reference to its defining class loader, and so it cannot be 
>> completely dereferenced either.
>>
>> To avoid this problem, any classes that are to be serialized should 
>> be loaded by a class loader that does not need to be garbage 
>> collected -- by the system class loader, for example.
>>
>>
>>
>
>