From dev-return-23885-apmail-harmony-dev-archive=harmony.apache.org@harmony.apache.org Sat Feb 03 14:04:25 2007 Return-Path: Delivered-To: apmail-harmony-dev-archive@www.apache.org Received: (qmail 82269 invoked from network); 3 Feb 2007 14:04:24 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 3 Feb 2007 14:04:24 -0000 Received: (qmail 93089 invoked by uid 500); 3 Feb 2007 14:04:30 -0000 Delivered-To: apmail-harmony-dev-archive@harmony.apache.org Received: (qmail 92520 invoked by uid 500); 3 Feb 2007 14:04:29 -0000 Mailing-List: contact dev-help@harmony.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@harmony.apache.org Delivered-To: mailing list dev@harmony.apache.org Received: (qmail 92510 invoked by uid 99); 3 Feb 2007 14:04:29 -0000 Received: from herse.apache.org (HELO herse.apache.org) (140.211.11.133) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 03 Feb 2007 06:04:29 -0800 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (herse.apache.org: domain of xiaofeng.li@gmail.com designates 66.249.92.169 as permitted sender) Received: from [66.249.92.169] (HELO ug-out-1314.google.com) (66.249.92.169) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 03 Feb 2007 06:04:19 -0800 Received: by ug-out-1314.google.com with SMTP id z36so853402uge for ; Sat, 03 Feb 2007 06:03:58 -0800 (PST) DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=uHhyYSDamdayT0k+c6J4d4BZw7GfBS5Zex776DnfxZL3R4asH/E2ST0EviyRsu69E+rOeGcAxK3q5Bpwc5ZaN78oIPw2Tkm/FM8IrYLmR22lEyv2ADXYBZ83ztPL3kPRxJ2lasc4BUsRAB0Mx44m66LUEUMFI/nl7t+Yo52luHM= Received: by 10.78.149.13 with SMTP id w13mr872331hud.1170511438211; Sat, 03 Feb 2007 06:03:58 -0800 (PST) Received: by 10.78.154.18 with HTTP; Sat, 3 Feb 2007 06:03:58 -0800 (PST) Message-ID: <9623c9a50702030603l200ff60chc5df802dc1a3a9e5@mail.gmail.com> Date: Sat, 3 Feb 2007 22:03:58 +0800 From: "Xiao-Feng Li" To: dev@harmony.apache.org Subject: Re: [VM]How to trigue GC while free native memory is low. In-Reply-To: <12385bbd0702030500g54a20475ke2cdefaa063337ab@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-2022-JP; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <9623c9a50702020105l7f070770m35594455bbfd6867@mail.gmail.com> <9623c9a50702020244p4bba7163m801286d46851212c@mail.gmail.com> <5c8e69f0702021902q1119434aqa8dd7b8b4bd0f9ff@mail.gmail.com> <12385bbd0702030500g54a20475ke2cdefaa063337ab@mail.gmail.com> X-Virus-Checked: Checked by ClamAV on apache.org On 2/3/07, Ivan Volosyuk wrote: > On 2/3/07, LvJimmy，Jing wrote: > > 2007/2/2, Xiao-Feng Li : > > > Thank you, Jimmy and Leo, for the clear explanations. > > > > > > I think temporarily you can put the System.gc() there to workaround > > > this issue. Yes, this is only a workaround since we cannot rely on it > > > without explicit guarantee in its semantics. > > > > > > To really solve this problem, we might need a careful design. Your > > > proposed approach looks like a possible solution. Have you thought to > > > solve the problem by using GC to manage the raw memory blocks? Without > > > deep understanding of the problem, I have some wild idea below. Please > > > see if it is reasonable or stupid. I will read the spec of ByteBuffer > > > to get more understanding. > > > > > > Since what the ByeBuffer needs are starting address and capacity, it > > > doesn't really care if this piece of memory is in Java heap or in > > > native runtime memory ( I assume.) We probably can provide some > > > special kind of virtual Java object that serves as raw memory block to > > > nio. In this works, we need not monitor the native runtime memory > > > usage. > > > > > > > Ah, this solution sounds interesting to me, yes, if we malloc memory > > in heap then we no longer need to monitor the native memory. But we > > may need a kit of methods offer this heap-memory service: > > 1) a function of malloc_in_heap() > > 2) a function of free_heap_block() > > 3) heap monitor should record this malloc/free action > > 4) ... > > However I do think this may be a good solution, and it still depends > > on how easy can it be implemented. Note not only direct ByteBuffer > > require this native memory usage. > > I like this idea. I think we can express it in more general way. We > need native memory which is somehow related to java heap objects. When > calling garbage collection there is high possibility that some of this > memory will be freed. So here is my variant of this proposal: > 1. a function gc_native_malloc() which allocates native memory and > triggers GC on a certain threshold. > 2. a fenction gc_native_free() which deallocates native memory and > updates counter for the used space. > This memory may be allocated in general native heap but it should be > accounted by GC. It is not even required that it should be the GC who > implements this functionality, just some code which will can call GC > and able to allocate native memory. > > One drawback of this approach that we should know all the places where > we have a relation between java heap object and native memory. Some > custom native library may disobey this rule. I've just studied ByteBuffer and MappedByteBuffer. If I understand correctly, the latter one is similar to POSIX mmap, and it's a subclass of ByteBuffer and uses a direct ByteBuffer. If it's implemented in the way of mmap, we probably have no serious issue to provide the required support in GC. The advantage to provide this support in GC is, GC can decide when and how to start a collection. And if the GC has some mechanism of reference counting, it can serve the purpose even better. The other point is, in the spec, the ByteBuffer can be created by wrapping a Byte array. GC can provide this buffer as a direct buffer if that's really wanted in certain scenario. Thanks, xiaofeng