From dev-return-33202-apmail-harmony-dev-archive=harmony.apache.org@harmony.apache.org Wed Apr 02 10:26:45 2008 Return-Path: Delivered-To: apmail-harmony-dev-archive@www.apache.org Received: (qmail 72873 invoked from network); 2 Apr 2008 10:26:42 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 2 Apr 2008 10:26:42 -0000 Received: (qmail 33226 invoked by uid 500); 2 Apr 2008 10:26:41 -0000 Delivered-To: apmail-harmony-dev-archive@harmony.apache.org Received: (qmail 33196 invoked by uid 500); 2 Apr 2008 10:26:41 -0000 Mailing-List: contact dev-help@harmony.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@harmony.apache.org Delivered-To: mailing list dev@harmony.apache.org Received: (qmail 33187 invoked by uid 99); 2 Apr 2008 10:26:41 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 02 Apr 2008 03:26:41 -0700 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of aleksey.shipilev@gmail.com designates 64.233.166.179 as permitted sender) Received: from [64.233.166.179] (HELO py-out-1112.google.com) (64.233.166.179) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 02 Apr 2008 10:26:00 +0000 Received: by py-out-1112.google.com with SMTP id a25so3482049pyi.13 for ; Wed, 02 Apr 2008 03:26:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; bh=yp+SBUHJQUYVTiLHlHl574Gom+eTJY2zrWs1GjBTOmU=; b=PMEGe54yUW8YySOxlIbv6+uZ5s5GvW4AYM2xn6UNyErPNGrJmnxvSzHa7G8oQy31/Z0WiqYWZ9RzfRkXYsMmMq/fkfP8GB5ZhAHoiGXZpOOI3R6zWG0Bt1YSWHrufGAGTztu8QeI8vggzwjWOgS2wh8eMURs6qP6gp1HQnGzpu4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=message-id:date:from:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=RQyNBlr10JxJe6DKLmjtQBDqsAaNXwMfEVgOtao/5KhBzzB7t7IABQY+elU9x4QGkOw5hWdGX4XOuGHmH3/H17CSmGTfjxSI4oWKv6IZFmHUR2eLWVH9tUNwtwK4Q4qGtcEB/3kq0zaIIzyvZS7+cg30oV4tOZ12v4GM4mCw/0c= Received: by 10.141.3.17 with SMTP id f17mr4845228rvi.180.1207131971613; Wed, 02 Apr 2008 03:26:11 -0700 (PDT) Received: by 10.141.142.20 with HTTP; Wed, 2 Apr 2008 03:26:11 -0700 (PDT) Message-ID: <4bebff790804020326n2184e868x83d237bae02c972b@mail.gmail.com> Date: Wed, 2 Apr 2008 14:26:11 +0400 From: "Aleksey Shipilev" To: dev@harmony.apache.org Subject: Re: [GSoC] Appliance for harmony-gc-4 "Unify the native memory management of Harmony DRLVM" In-Reply-To: <9623c9a50803311851m6322263aj1b5a65364261ae75@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <4bebff790803271010t7912b1b7r77ebd2a5ad5c6e08@mail.gmail.com> <4bebff790803300753qe3bbcfau64ad46928d77d266@mail.gmail.com> <4bebff790803301409h11e6afe4k8d98e2fa9d2045ef@mail.gmail.com> <4bebff790803310652n5d7fc059m3726144deb739960@mail.gmail.com> <9623c9a50803311851m6322263aj1b5a65364261ae75@mail.gmail.com> X-Virus-Checked: Checked by ClamAV on apache.org Thanks, Andrey, Xiao Feng! Your comments are very valuable. 1. I had updated the wiki [1] with new version of proposal, emphasized once more on goals. 2. Metric of success. I think that (a) projected boosts could not be estimated right now, without working prototype, (b) bugs in memory management are also covered by multiple wrappers and I expect some of them will be exposed after moving to UMM, but still it's hard to reveal without working prototype. So, I choose the "UMM is ready and used in most of VM components" as good enough starting metric. What do you think, Andrey? 3. GC and mmap. Though I had little experience with GC heap adjustment mechanisms, why can't GC just use hyvmmap wrapper for mmap? If we will cover the same interface as mmap do, then there should be no problems, right? Moreover, it can eliminate cross-platform memory management wrappers inside the GC and delegate this to UMM. Xiao Feng, what do you think? Thanks, Aleksey. [1] http://wiki.apache.org/general/AlekseyShipilev/GSoC2008/harmony-gc-4 On Tue, Apr 1, 2008 at 5:51 AM, Xiao-Feng Li wrote: > Hi, Aleksey S. very good proposal. It deserves Google support. :-) > > Two comments: > 1. goal of the project. Your stated goals are related only to software > engineering issues. I think it should also be related to performance > issue and research issue. Specifically, a unified native MM can open > up new opportunities such as large page for the entire system, data > prefetch, JIT code caching management, Java directBuffer improvement, > etc. > 2. GC related. GC uses mmap for good reasons. It need to map certain > pages or unmap them sometimes. This support is important for GC heap > adjustment. It's common for GC to reserve certain space, then commit > or decommit part of it. So basically GC has two set of memory uses: > one is like malloc for its own runtime allocations, the other is like > mmap for Java heap management. I would suggest we ignore the latter > category at the beginning of the project. It's not an easy stuff until > we have good understandings about the unified NMM. > > Thanks, > xiaofeng > > On Mon, Mar 31, 2008 at 9:52 PM, Aleksey Shipilev > > wrote: > > > > Alexei, > > > > I had addressed both of your issues in the updated proposal [1]: > > a. Added portlib pools as the candidate. > > b. Rephrased abstract a little, with emphasis on unifying. > > c. "Class-based service" added to improvements plan: service critical > > customers like exception handlers, etc. > > d. "Wrapping malloc/free" added to improvements plan. > > > > I want to thank you with the idea (d) - that's the way of moving > > Classlib native code to UMM. > > > > Xiao Feng, Andrey (Yakushev), can you please review? > > > > > > Thanks, > > Aleksey. > > > > [1] http://wiki.apache.org/general/AlekseyShipilev/GSoC2008/harmony-gc-4 > > > > > > > > On Sun, Mar 30, 2008 at 10:01 PM, Alexei Fedotov > > > > wrote: > > > > > > > > > You forgot portlib pools. All this reads in a following way "We have > > > > > seven memory subsystems, and I want to implement the eighth which > > > > > would be the best." Ok, I put the same intention into STD_MALLOC three > > > > > years ago. We attached APR for the same reason.You suggest adding UMM. > > > > > Amount of systems continues to grow. Contrary, I would suggest having > > > > > less memory management subsystems on completion of your project. > > > > > > > > > > BTW, two important use cases are missed in your proposal. The native > > > > > memory subsystem should continue serving critical customers such as > > > > > lazy exception messages or finalizers even when it reported to others > > > > > that the memory is exhausted. To prevent user's JNI code from > > > > > exhausting memory we did not find a solution other than substitution > > > > > of function table in C runtime library with our functions. Redirecting > > > > > calls to our functions allows us plugging OS-level optimized memory > > > > > managers and in this case we can continue using conventional malloc > > > > > and free rather than inventing hymalloc. > > > > > > -- > http://xiao-feng.blogspot.com >