harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tim Ellison <t.p.elli...@gmail.com>
Subject Re: [classlib] Code coverage
Date Fri, 24 Aug 2007 09:43:05 GMT
Leo Li wrote:
> On 8/23/07, Rui Hu <roberthurui@gmail.com> wrote:
>> I've shared the customized emma tool and its usage in [1].
>>
>> We can also find the reason why we need customize emma and how to do it.
>> The
>> modified source code is also provided.
>>
>> Any comment is highly welcomed.
>>
>> [1]: http://wiki.apache.org/harmony/coverageEMMA
> 
> 
>    The existence of kernel list is, as Robert says, because that Emma itself
> depends on java core classes in its static initialization but
> unfortunately in order to record testcoverage of these java core classes, in
> their static initialization, emma will be referenced in order to register
> some data structure for these java core classes. Thus Emma' initialization
> depends on java core classes and java core classes' initialization depends
> on Emma and VM detects a circular dependency and throw out ClassDefNotFound
> exception.
>    In out current solution, core classes in the kernel list will not
> register themselves to emma during instrumentation. But during startup,  we
> will explicitly load these core classes and Emma  and then manually register
> core classes afterwards.
>    But what puzzles me is that the kernel list, which is the dependencies of
> Emma(The classes will be loaded when loading Emma.), is so unstable and is a
> lot larger than my estimation. I once used JVMTI to record all the classes
> loaded when loading Emma,  and the kernel list generated by this way is much
> smaller and does not work.
>   So is there a more precise way to calculate which class will be referenced
> while loading Emma(The com.vladium.emma.rt.RT class)? I believe the
> dependencies of this class should be more stable than we have met.

How about "java -verbose:class ..."?  That will tell you which classes
are loaded.  But you should have been able to get that info via JVMTI so
maybe I misunderstand.

Regards,
Tim

> 2007/8/22, Rui Hu <roberthurui@gmail.com>:
>>> I've shared the coverage report in HARMONY-4665. Anyone can download it
>> to
>>> see the coverage details. Thanks.
>>> The customized EMMA and usage will be shared later.
>>>
>>>
>>> 2007/8/21, Tim Ellison < t.p.ellison@gmail.com>:
>>>> Rui Hu wrote:
>>>>> I think, it is not easy to automatically generate the coverage
>> report
>>>> of our
>>>>> trunk. :-(
>>>>>
>>>>> Because Harmony is not a Java application, it's the JDK, so the
>>>> original
>>>>> EMMA can not be use to collect the coverage data of Harmony. We've
>>>> modified
>>>>> the source code of EMMA, and realize the coverage collecting for
>>>> core-java
>>>>> APIs. In this way, we must notice that:
>>>>> 1. The coverage of the classes in luni-kernel.jar can not be
>>>> collected. It's
>>>>> not a big problem, because there're not many classes in that jar and
>>>> the
>>>>> coverage of those classes is considered to be very high (they're
>>>> usually
>>>>> used kernel classes).
>>>>> 2. The customized EMMA needs extra list of booting classes to run,
>> the
>>>> class
>>>>> list can not be too big or too small. If it's too big, the EMMA may
>>>> not
>>>>> work. If it's too small, the JDK can not initialize.
>>>>> 3. The coverage of the classes in the list mentioned in #2 will be
>>>> always
>>>>> 100%, (only the class coverage will be 100%, the method/block
>> coverage
>>>> will
>>>>> be correct), it's reasonable for the REAL booting classes, but
>>>> incorrect for
>>>>> the unnecessary classes. So, this is another reason why we need not
>> a
>>>>> too-big list.
>>>>>
>>>>> The problem is from the booting classes list mentioned in #2.
>> Ideally,
>>>> this
>>>>> list includes the classes which is used during initialization of
>>>> running a
>>>>> test suite. Unfortunately, we do not have method to get the list
>>>>> automatically, and it related with the revision of Harmony build,
>> the
>>>>> classes we want to collect coverage data for, and the test suite we
>>>> want to
>>>>> run.
>>>>> So we now must try to get the list from cutting down the big-enough
>>>> list
>>>>> (usually the all-instrumented-classes-list) every time. If the test
>>>> crash,
>>>>> we must modify the booting classes list or remove some "problem test
>>>> cases".
>>>>> The only shortcut is use the current successful list as the start
>>>> point, but
>>>>> it can not ensure the test will run successfully, especially we
>> change
>>>> the
>>>>> test suite or expand the instrumented classes list ( for example,
>>>> instrument
>>>>> all the classes instead of only public API classes).
>>>>>
>>>>> I'd like to share our customized EMMA and the experience of using
>> it.
>>>> It is
>>>>> highly welcomed to share your idea to automatize this process, or
>>>> simplify
>>>>> the it, or make the it more stably.
>>>> It would be great if you would share it, then perhaps other people
>> will
>>>> see ways to improve creating the booting classes list -- and it would
>> be
>>>> interesting to get reports on the current BTI test coverage.
>>>>
>>>> Regards,
>>>> Tim
>>>>
>>>
>>>
>>> --
>>> Robert Hu
>>> China Software Development Lab, IBM
>>>
>>
>>
>> --
>> Robert Hu
>> China Software Development Lab, IBM
>>
> 
> 
> 

Mime
View raw message