Mailing-List: contact dev-help@harmony.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@harmony.apache.org
Received-SPF: pass (athena.apache.org: domain of t.p.ellison@gmail.com
 designates 209.85.219.20 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=message-id:date:from:user-agent:mime-version:to:subject:references
         :in-reply-to:x-enigmail-version:content-type
         :content-transfer-encoding;
        b=ksnyqub30Y3GFIe4cpUE9WgF6rwgvh9KGeL20fZydrE/Ljz3zU4YdIn5EJJN62FB+Q
         7iFCqZmDpZpljPvFOzldlFRWoh8rPAXvVW7h6ZRIC/xjdFkSzmbNLy8LWolTr7T1wcoC
         ugZXlfbImn3auBTnwjRjny2ErEDIG1nPTRnNc=
Message-ID: <493FC000.6010708@gmail.com>
Date: Wed, 10 Dec 2008 13:11:28 +0000
From: Tim Ellison <t.p.ellison@gmail.com>
User-Agent: Thunderbird 2.0.0.18 (Windows/20081105)
MIME-Version: 1.0
To: dev@harmony.apache.org
Subject: Re: [vm] Implement on-demand class library parsing to avoid
 unnecessary
 jar/zip parsing during startup (HARMONY-6039)
References: <1682519387.1228881884465.JavaMail.jira@brutus>
	 <493F9EE6.8010704@gmail.com>
 <9623c9a50812100450y10af157egb7938cde50872781@mail.gmail.com>
In-Reply-To: <9623c9a50812100450y10af157egb7938cde50872781@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

Xiao-Feng Li wrote:
> On Wed, Dec 10, 2008 at 6:50 PM, Tim Ellison <t.p.ellison@gmail.com> wrote:
>> This is cool.
>>
>> Wenlong Li (JIRA) wrote:
>>> During VM creation, Harmony will parse all class libraried defined in
>>> bootclasspath.properties under jre/lib/boot directory. However, not
>>> all class libraries are accessed during startup. That means, it is
>>> not necessary to open and resolve all these class libraries.
>> Agreed.
>>
>>> In this patch, I implement the on-demand jar parsing. I leverage the
>>> class library info defined in the manifest file of each module. For a
>>> request class, if it is not available in existing class table, I then
>>> parse its class library info, and check which module contains this
>>> requested class. That is, I parse the class library on demand.
>> Its good to see the manifest information being used at runtime.
>>
>> So as I understand it, you are not looking to provide any isolation
>> between modules, just using the metadata to direct the search path.
>>
>>> Using on-demand class library parsing, I can reduce the VM creation
>>> time from 20+ seconds to 3 seconds.
>> Wow, that's remarkable.  Is ZIP file directory performance really that bad?
>>
>> I'm assuming that it is multiple VM creations (it doesn't take 20s to
>> invoke the VM usually!)  And that is still opening and reading the
>> manifest from each JAR?
> 
> Cold start (first time launch) and warm start (not first time launch)
> have very different numbers, because of the disk access caching
> effect. The numbers in seconds should be cold start. I remember in
> Wenlong's data given early, this work can improve the warm start time
> as well, something like from 170ms to 140ms.

Luckily, I've never seen a 20s start up time.

So my question is where is the time being saved using this technique?

Is it the number of files we touch? e.g. disc seek time, or is it
searching through the ZIP directories looking for class files e.g. poor
ZIP performance for our usage, or ...

Given that the cold start times improve by 85% (20s down to 3s), and
warm start improves by ~18% (170ms down to 140ms), it is most likely the
'getting the bits off disc' part.  Does that sound right?  Can we
measure it?

>> I assume we could improve that by consolidating that metadata into a
>> file with faster access.
> 
> Yes, below is what Wenlong's patch includes.
> 
> +# The modulelibrarymapping properties show class libraries in each module.

I saw that, but not sure which problem it is solving
(seek times/parsing/other).  Also, of course, this is a copy of the
metadata that we would need to keep in sync with changes to the module
manifests.  I'd like the best of both worlds :-)

Regards,
Tim