harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wenlong Li" <wenl...@gmail.com>
Subject Re: [vm] Implement on-demand class library parsing to avoid unnecessary jar/zip parsing during startup (HARMONY-6039)
Date Thu, 11 Dec 2008 02:49:57 GMT
all,

Thx for your kind suggestion.

The performance of on-demand library parsing is mostly from I/O (seek
the jar, open the jar, read content for each class in the jar, etc).
When disk cache is warm-up, the I/O impact is small, and then we see
small performance improvement.

You see the bootclasspath property is pre-defined in
working_classlib\depends\files, so I pre-define another property file
which saves the class library info for each package. I assume the
library in each module will not change over time, just like the
bootclasspath property file. Of course, when new module is added, we
need update these properties. Question is how Harmony revises the
bootclasspath property now?

I will think about how to automatically extract the package info from
manifest of each module, and save them in my defined property file at
compiling time.

Suggestion and comments are welcome and appreciated. :)

On Thu, Dec 11, 2008 at 10:12 AM, Nathan Beyer <ndbeyer@apache.org> wrote:
> I like the proposal, but I'd like to see an addition that either
> negates the duplication of manifest information or automates the
> generation of it. The later may be easiest - add a bit to the build
> that reads the manifests and generates the file.
>
> -Nathan
>
> On Wed, Dec 10, 2008 at 8:22 AM, Xiao-Feng Li <xiaofeng.li@gmail.com> wrote:
>> On Wed, Dec 10, 2008 at 10:07 PM, Tim Ellison <t.p.ellison@gmail.com> wrote:
>>> Xiao-Feng Li wrote:
>>>> On Wed, Dec 10, 2008 at 9:11 PM, Tim Ellison <t.p.ellison@gmail.com>
wrote:
>>>>> So my question is where is the time being saved using this technique?
>>>>>
>>>>> Is it the number of files we touch? e.g. disc seek time, or is it
>>>>> searching through the ZIP directories looking for class files e.g. poor
>>>>> ZIP performance for our usage, or ...
>>>>>
>>>>> Given that the cold start times improve by 85% (20s down to 3s), and
>>>>> warm start improves by ~18% (170ms down to 140ms), it is most likely
the
>>>>> 'getting the bits off disc' part.  Does that sound right?  Can we
>>>>> measure it?
>>>>
>>>> I am also wondering about it. There are two ways to save time: one is
>>>> to reduce the number of JARs by repackaging all the classes; the other
>>>> is to reduce the number of JARs loaded. As I know, the first approach
>>>> helps little. That probably means the disk seeking time is the major
>>>> contributor of startup time.
>>>>
>>>> If this hypothesis is correct, then to delay the loading of JARs can
>>>> benefit even if all the JARs are needed in the end by an application.
>>>> The reason is the IO time can be overlapped with the computation time.
>>>
>>> Yes, especially if we have the metadata that tells us where the classes
>>> are stored.  We can queue requests to load multiple class files
>>> asynchronously in the file system.
>>>
>>>> Well,  this solution might not really help, if an application (or
>>>> benchmark) only measures the period after VM is launched. (This is
>>>> probably true since a Java timer can be triggered only after VM is
>>>> created, unless the benchmark measures the duration of twice VM
>>>> instance creations.)
>>>
>>> Agreed, we should agree what we are optimizing, and why (the use case).
>>>
>>>>>>> I assume we could improve that by consolidating that metadata
into a
>>>>>>> file with faster access.
>>>>>> Yes, below is what Wenlong's patch includes.
>>>>>>
>>>>>> +# The modulelibrarymapping properties show class libraries in each
module.
>>>>> I saw that, but not sure which problem it is solving
>>>>> (seek times/parsing/other).  Also, of course, this is a copy of the
>>>>> metadata that we would need to keep in sync with changes to the module
>>>>> manifests.  I'd like the best of both worlds :-)
>>>>
>>>> Right. A "better" solution I can think of is, the runtime can extract
>>>> this information at build time or installation time. It can be the
>>>> next step if we think it is a valid solution.
>>>
>>> The OSGi frameworks must have this problem already.  They need to
>>> maintain a cache of bundles installed in the runtime without it being a
>>> surprise each time they start up.  While Wenlong's static cache is fine
>>> for the prototype, there may be some code in Felix/whatever that we can
>>> use to dynamically manage cached metadata for our modules.
>>
>> Agreed. This kind of dynamic Felix-based way probably will be a long
>> term ultimate solution.
>>
>>> p.s. Please don't take my comments as negative about the approach here.
>>>  I'm delighted to see this type of innovative work taking place.
>>
>> Definitely not negative. Comments are always welcome. And I think
>> Harmony community needs more comments. :)
>>
>>> Regards,
>>> Tim
>>>
>>>
>>
>>
>>
>> --
>> http://xiao-feng.blogspot.com
>>
>

Mime
View raw message