Mailing-List: contact dev-help@harmony.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@harmony.apache.org
Received-SPF: pass (herse.apache.org: domain of firepure@gmail.com designates
 64.233.182.184 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=beta;
        h=received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references;
        b=QuOEO3tEODy4xSfQ6DBo3C4hLclK56FessnL5BmVYI0n9t2MzWTLWz3i4OJDlbD2oMuyCGd0ULwZbmYGtuaWqzPVqEyewn2wc6VPSrJ8QMPV7S4WKwCMm+Z5170pO+HCiYR/jBQPh7jilPExVAP8KwPdaC48IWlhpdK0EfxLMNs=
Message-ID: <5c8e69f0702020138q73150c86ha78d969aceee7d58@mail.gmail.com>
Date: Fri, 2 Feb 2007 17:38:18 +0800
From: "=?GB2312?B?THZKaW1teaOsSmluZw==?=" <firepure@gmail.com>
To: dev@harmony.apache.org
Subject: Re: [VM]How to trigue GC while free native memory is low.
In-Reply-To: <9623c9a50702020105l7f070770m35594455bbfd6867@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
References: <e66844de0702012308r619847e4wc66f692d7ac67a8b@mail.gmail.com>
	 <9623c9a50702020105l7f070770m35594455bbfd6867@mail.gmail.com>

2007/2/2, Xiao-Feng Li <xiaofeng.li@gmail.com>:
> Leo, I can't fully understand your problem and proposal. I am working
> in DRLVM GC component, I want to understand how GC can tackle the
> issue you meet.
>
> You said the native memory is allocated (probably with malloc) for
> byte buffer. I wonder how the process goes. I assume it's in native
> code. Then you use malloc to allocate a block of memory, and put some
> Java object (byte buffer) into it? Is this what you do? I wonder how
> do you manipulate Java object placement in native code; and if the
> native memory block is out of Java heap, do you expect GC to manage
> it?
>
> You said when GC reclaims those byte buffer object, the native memory
> block can be freed in native code. How do you know if an object is
> reclaimed by GC or not?
>

Hi,
     What Leo is talking about is DirectByteBuffer, which is added in
Java 5. This class is very strange compared to early classes. In java5
,we can allocate "direct" java.nio.ByteBuffer referring to the block
of memory starting at the memory address address and extending
capacity bytes. This block of memory is not in java heap so that it
can not be control by GC. However, we have a ByteBuffer object in java
heap(referring to that block of memory), if we gc this ByteBuffer
object, the VM also free that block of memory(This prevents memory
leaking).
      But if we have a lot of un-used ByteBuffer(e.g, 100) in java
heap, they may refer to much native memory(e,g, each of them own a
memory block of 10m),  so we must have 1G native memory in use, but at
that time, java-heap may be not full(only 100+ objects in heap), the
GC will not invoke, but real native memory will be full, so next time
we try to new a DirectByteBuffer will cause a memory-out error, which
can be avoid if we ask GC to invoke and clear these 100 unused
ByteBuffers and VM will free 1G memory.
      So to conclude this question, we should invoke GC when we find
there's too much native memory in use(not how GC can handle this
problem), so
1) how to know there's too much?
2) how to invoke GC when we need to?
And my question is 3) where is suitable to put this part of codes?

Leo may offer some simple sample for this?

> Thanks,
> xiaofeng
>
> On 2/2/07, Leo Li <liyilei1979@gmail.com> wrote:
> > Hi, all:
> >     After applying patch3073, I added the support for direct byte buffer in
> > nio module as spec requires. But now here exists a problem: The native
> > memory allocated to the byte buffer can be released when gc releases the
> > byte buffer object, however, if the native heap is full while java heap
> > still has space, gc will not be triggered.
> >     It seems that RI will start gc before native memory heap is depleted and
> > thus prevents out-of-memory error.
> >
> >    Then our work focuses on:
> >    1. When gc is required.
> >    2. Trigger a gc.
> >    The first one requires that we get support from operating system, since
> > the memory allocated in the native code, for example by malloc, is out of
> > the control of java VM.( I have ever thought of counting the used memory in
> > hymemory_allocate, but the plan fails since hymemory_free will not report
> > how much space is released.)
> >    The second one needs the help from VM. System.gc() is not so reliable as
> > spec says, so it is necessary to have a internal channel to notify VM to
> > start gc.
> >
> >     One solution, I think, is to let a monitor thread in VM to check whether
> > OS physical memory is low. For example, the QueryMemoryResourceNotification
> > of win32 API is a candidate. Although the interrupt model is more effective:
> > win32 SDK provides a CreateMemoryResourceNotification to get a handler on
> > which the monitor thread can wait, maybe on other platforms, OS does not
> > supply such a convenience. So the monitor thread in the VM might have to
> > check the OS resource once for a while and if necessary the monitor thread
> > will call a GC.
> >
> >    My suggestion is first to add some function to monitor memory in portlib,
> > since it is highly related to platforms and in portlib there has been
> > some useful tools.(Thanks Mark to point out that.) On the other hand, we can
> > negotiate a channel to trigger gc in VM. Actually I am not an expert on VM
> > since I am not sure whether there has been some monitor thread and the load
> > on performance if such a monitor thread is added to the VM...
> >
> >
> > --
> > Leo Li
> > China Software Development Lab, IBM
> >
> >
>


-- 

Best Regards!

Jimmy, Jing Lv
China Software Development Lab, IBM