Mailing-List: contact dev-help@harmony.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@harmony.apache.org
Received-SPF: pass (athena.apache.org: domain of simon.harmony@gmail.com
 designates 209.85.212.182 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :cc:content-type;
        b=Zlt7//CBplQmBoTGJIYI6n9O4RyW1re6j/EC3ATZXTx+hVNElNZnzLldHLsgBvh40M
         NSUDyUmDjYr836e24Yt3X5Mimu/71ScPybth/xa83nTZNeZBLfje8XM0nu4MJe6xAxid
         LyEenuTAcks/M9iqaCA7tJafg84tpmqqh8ESM=
MIME-Version: 1.0
In-Reply-To: <9623c9a50908200150o76101027g6e9a7f2a8a41a825@mail.gmail.com>
References: <9623c9a50908041855k2cb46690nc0acab1204c597f3@mail.gmail.com>
	 <3db9f87f0908050003h56d055ew58351fd3ee76f3ac@mail.gmail.com>
	 <9623c9a50908050227s142f6199n108a20ea57c7b8b6@mail.gmail.com>
	 <3db9f87f0908092150t3cd28734n50fe1d4410567a7c@mail.gmail.com>
	 <9623c9a50908100547q34a780dbh96e85cfad0522eac@mail.gmail.com>
	 <9623c9a50908162055n4598e889sa9c95cc8da8a7a04@mail.gmail.com>
	 <3db9f87f0908170747r1f8008b7l270ce6c3cc6c6d94@mail.gmail.com>
	 <9623c9a50908200150o76101027g6e9a7f2a8a41a825@mail.gmail.com>
Date: Fri, 21 Aug 2009 10:55:47 +0800
Message-ID: <3db9f87f0908201955g64aa3cf8hdc043f5dbcb5718@mail.gmail.com>
Subject: Re: [GSoC] time to close
From: Simon Zhou <simon.harmony@gmail.com>
To: Xiao-Feng Li <xiaofeng.li@gmail.com>
Cc: dev@harmony.apache.org
Content-Type: multipart/alternative; boundary=0016e64ec5104d69b404719e0167

--0016e64ec5104d69b404719e0167
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

Hi, Xiaofeng,

For now, just turn on USE_UNIQUE_MARK_SWEEP_GC in gc_common.h will be OK,
because the gc.ignore_finref is false by default.
On the contrary, to turn off the weak reference supporting, add
-XX:gc.ignore_finref=true in the command line will work.
For the test cases, I am sorry I have not special test cases for this
project, I just using specjbb and dacapo, and exam the dump information to
perform testing.
I think these tests can cover most cases, except they do not strickly check
the enqueue operations.

I thought about how to write java code test to strickly check the enqueue
operations in Concurrent GC, but I found it is a little hard. The reason is
that, the application code can not know the start time and end time of
Concurrent GC, while the enqueue operations only should be performed after
GC.

It is a little different from the case of STW GC, in smoke test, application
code can just call System.gc() to force a STW GC, and then check if the
enqueue operations are right. In such a case, if the referent become
NULL and enqueue operations are not performed, it could be regarded as a GC
fault (Or not?). But in current design of Concurrent GC, this may really
happend because the enqueue operations happens in the end of GC, while 'set
NULL' happens in the process of GC.

So, IMHO, there are 2 options to solve this issue, one is change the current
design, make the 'set NULL' and 'enqueue' in a atomic transaction (which is
not in java spec [1]) which is transparent to the application code, but
it may impact the performance.
Another choice is that, write special elaborate test cases which waits some
manully-setting period after finding the referent is NULL, then check the
enqueue operations (at this time, Concurrent GC will hopfully be finished
and enqueue the references). But how long of the manully-setting period
will depend on the implementation.

Do you think this make sense? Thanks!

[1], http://java.sun.com/javase/6/docs/api/java/lang/ref/WeakReference.html<http://java.sun.com/javase/6/docs/api/index.html>

Thanks
Simon

2009/8/20 Xiao-Feng Li <xiaofeng.li@gmail.com>

> Simon, the summary looks good.
>
> Still, would you like to write a brief doc on  how to turn on your
> work and test it? Have you any special test cases developed for the
> project?
>
> Thanks,
> xiaofeng
>
> On Mon, Aug 17, 2009 at 10:47 PM, Simon Zhou<simon.harmony@gmail.com>
> wrote:
> > Hi Xiaofeng,
> >
> > I would like to give a brief summary of Concurrent GC weak reference
> > supporting project
> >
> >
> >
> > 1, Goal
> >
> >  The main goal of this project is to make the Harmony Concurrent GC
> process
> > weak/soft/phantom reference and finalizable objects properly. Weak
> reference
> > is in java spec and a important feature supported by java languages,
> > programmers can use them to give hint to JVM to manage their
> applications'
> > memory.
> >
> >
> >
> > 2, Design and implementation
> >
> > There are 2 major difference between weak references processing in
> > STW/Generational GC and that of Concurrent GC.
> >
> >
> >
> > First,
> >
> > we must intercept all the reference.get() invokes to remember the
> referent
> > whose reference is 'leaked out'.
> >
> >
> >
> > Second,
> >
> > the weak/soft reference processing may change the referent's
> reachability,
> > and does not change other part of the object referencing graph. So we can
> do
> > this job concurrently after marking is finished.
> >
> > On the other hand, phantom reference processing should be after
> finalizable
> > objects processing, while the later one will change the object
> referencing
> > graph. So we should do this job in a STW phase.
> >
> >
> >
> > For the first issue, it is necessary before we can move forward to next
> > steps., I use 2 different technicals to catch the get() method, one is
> JNI
> > implementation, which is straightforward and easy to implement, but has
> big
> > overhead because of the frequent JNI calls. this implementation can be
> > treated like a stepping-stone. The other one is a VMMagic version, which
> is
> > just like the write barrier implementation in current Concurrent GC. By
> > modify some passes of JIT, the dynamic compiler will generate a native
> call
> > to the barrier function when compiling java reference.get() method. this
> > version of barrier has a low overhead. now, it is default.
> >
> > For the second issue, which is the main part of this project. I uploaded
> a
> > doc to HARMONY-6258, it describe some details of implementation and I
> think
> > the diagram can explain better.
> >
> >
> >
> > 3, Result
> >
> > All the functionalities and thoughts mentioned in the proposal has been
> > implemented, and I have submitted the code patch to HARMONY-6258.
> >
> > Until now, I mainly used specJbb to evaluate the result, which uses
> > weak/soft/phantom reference and finalizable object when creating each of
> its
> > working houses. Besides I use dacapo to do some short-time-run test. I
> used
> > to dump the object info of get() barrier and reference processing phase.
> > They work well and correct, and the dump information is just as expected.
> >
> > As we know, the evaluation of weak reference processing should include
> the
> > enqueue operations. but until now, I have not found a long-time-run
> (because
> > it is a concurrent GC) benchmark which focus on automatically
> > and strictly examining when the referent should be dead (it is somehow
> > depend on different implementations) and if dead references are all
> execute
> > the weak reference enqueue operations properly. I do have planned to
> write
> > such a test suite and use it to evaluate my project, but it is
> > an elaborate work and is not ready yet. Sorry for this, I know it is
> > important, and I will finish it before any other improvements.
> >
> >
> >
> > 4, Future work and Further ideas
> >
> > After the implementation of this project, I am thinking of
> > some improvement I can do in the future.
> >
> > A,Benchmark or test suite which focus on weak/soft reference processing
> > (heap usages), checking the enqueue operations, finalizing and phantom
> > reference.
> >
> >
> >
> > B,Now, weak reference processing is done by the last concurrent marking
> > thread. When the weak references are only a very small part of the heap,
> > this works fine, but when there are lots of weak references to be
> processed
> > in the concurrent phase, the last marker's workload is heavy. To improve
> the
> > load balance, we need a parallel, concurrent weak reference processing
> phase
> > just after the concurrent marking.
> >
> >
> >
> > C,I am thinking of if it is possible to implement the resurrect phase of
> > finalizable objects processing in a concurrent phase. This may need a
> more
> > complicated write barrier which will only be 'turning on' in resurrect
> phase
> > and a some tracing work for the dirty objects.
> >
> > Thanks for your great mentoring!
> >
> > Thanks
> > Simon
> > 2009/8/17 Xiao-Feng Li <xiaofeng.li@gmail.com>
> >>
> >> Simon, today is for final "pencil down" and starting the GSoC project
> >> evaluation.
> >>
> >> I've read your doc in Harmony-6258, and reviewed the patch. They look
> >> good to me.
> >>
> >> One thing I want to know is, how you tested your work? what tests did
> >> you use to demonstrate your work is correct?
> >>
> >> Would you please write a doc on the steps to build and test your code?
> >> (and upload the test cases you developed?)
> >>
> >> Thanks,
> >> xiaofeng
> >>
> >> On Mon, Aug 10, 2009 at 8:47 PM, Xiao-Feng Li<xiaofeng.li@gmail.com>
> >> wrote:
> >> > Simon, thanks.
> >> >
> >> > I will review your patch and doc. If they are ok, I guess we can call
> >> > it a successful project. But the end goal is to commit the code
> >> > successfully. :)
> >> >
> >> > Thanks,
> >> > xiaofeng
> >> >
> >> > On Mon, Aug 10, 2009 at 12:50 PM, Simon Zhou<simon.harmony@gmail.com>
> >> > wrote:
> >> >> Hi Xiaofeng,
> >> >>
> >> >>   I have submitted a new patch for Concurrent Weak Reference project.
> >> >>   Now, Concurrent Weak Reference for mostly concurrent algorithm is
> >> >> ready.
> >> >>   Concurrent Weak Reference for STAB algorithm is almost ready, but a
> >> >> bug
> >> >> only when using concurrent mark & concurrent sweep running after
> dozens
> >> >> of
> >> >> GC iterations, I am checking this bug to find the root cause.
> >> >>   besides these, I am planning to write a simple doc for the
> >> >> implementation
> >> >> of concurrent weak reference and will submit with the patch.
> >> >>   Is there any other documentations I need provide for concluding
> this
> >> >> project? Thanks!
> >> >>
> >> >> Thanks
> >> >> Simon
> >> >> 2009/8/5 Xiao-Feng Li <xiaofeng.li@gmail.com>
> >> >>>
> >> >>> Simon, thanks for the update.
> >> >>>
> >> >>> Thanks,
> >> >>> xiaofeng
> >> >>>
> >> >>> On Wed, Aug 5, 2009 at 3:03 PM, Simon Zhou<simon.harmony@gmail.com>
> >> >>> wrote:
> >> >>> > Hi Xiaofeng,
> >> >>> >
> >> >>> >   Now, I have completed the WeakReference Processing (get barrier,
> >> >>> > weakref,
> >> >>> > softref, finalizable, phantomref) for mostly concurrent algorithm.
> >> >>> >   For STAB algorithms' implementation is still in debugging (get
> >> >>> > barrier,
> >> >>> > weakref, softref is OK).
> >> >>> >   So I will send a patch included mostly concurrent algorithm
> >> >>> > implementation
> >> >>> > first.
> >> >>> >
> >> >>> > Thanks
> >> >>> > Simon
> >> >>> >
> >> >>> > 2009/8/5 Xiao-Feng Li <xiaofeng.li@gmail.com>
> >> >>> >>
> >> >>> >> Hi, Simon, it is time to conclude your project soon. It is
> probably
> >> >>> >> good for you to let the community know your current status.
> >> >>> >>
> >> >>> >> Thanks,
> >> >>> >> xiaofeng
> >> >>> >>
> >> >>> >> --
> >> >>> >> http://people.apache.org/~xli
> >> >>> >
> >> >>> >
> >> >>> >
> >> >>> > --
> >> >>> > From : Simon.Zhou@PPI, Fudan University
> >> >>> >
> >> >>>
> >> >>>
> >> >>>
> >> >>> --
> >> >>> http://people.apache.org/~xli
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> From : Simon.Zhou@PPI, Fudan University
> >> >>
> >> >
> >> >
> >> >
> >> > --
> >> > http://people.apache.org/~xli
> >> >
> >>
> >>
> >>
> >> --
> >> http://people.apache.org/~xli
> >
> >
> >
> > --
> > From : Simon.Zhou@PPI, Fudan University
> >
>
>
>
> --
>  http://people.apache.org/~xli
>


-- 
>From : Simon.Zhou@PPI, Fudan University

--0016e64ec5104d69b404719e0167--