harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rui Hu" <roberthu...@gmail.com>
Subject Re: [classlib] Code coverage
Date Wed, 22 Aug 2007 07:26:34 GMT
I've shared the coverage report in HARMONY-4665. Anyone can download it to
see the coverage details. Thanks.
The customized EMMA and usage will be shared later.


2007/8/21, Tim Ellison <t.p.ellison@gmail.com>:
>
> Rui Hu wrote:
> > I think, it is not easy to automatically generate the coverage report of
> our
> > trunk. :-(
> >
> > Because Harmony is not a Java application, it's the JDK, so the original
> > EMMA can not be use to collect the coverage data of Harmony. We've
> modified
> > the source code of EMMA, and realize the coverage collecting for
> core-java
> > APIs. In this way, we must notice that:
> > 1. The coverage of the classes in luni-kernel.jar can not be collected.
> It's
> > not a big problem, because there're not many classes in that jar and the
> > coverage of those classes is considered to be very high (they're usually
> > used kernel classes).
> > 2. The customized EMMA needs extra list of booting classes to run, the
> class
> > list can not be too big or too small. If it's too big, the EMMA may not
> > work. If it's too small, the JDK can not initialize.
> > 3. The coverage of the classes in the list mentioned in #2 will be
> always
> > 100%, (only the class coverage will be 100%, the method/block coverage
> will
> > be correct), it's reasonable for the REAL booting classes, but incorrect
> for
> > the unnecessary classes. So, this is another reason why we need not a
> > too-big list.
> >
> > The problem is from the booting classes list mentioned in #2. Ideally,
> this
> > list includes the classes which is used during initialization of running
> a
> > test suite. Unfortunately, we do not have method to get the list
> > automatically, and it related with the revision of Harmony build, the
> > classes we want to collect coverage data for, and the test suite we want
> to
> > run.
> > So we now must try to get the list from cutting down the big-enough list
> > (usually the all-instrumented-classes-list) every time. If the test
> crash,
> > we must modify the booting classes list or remove some "problem test
> cases".
> >
> > The only shortcut is use the current successful list as the start point,
> but
> > it can not ensure the test will run successfully, especially we change
> the
> > test suite or expand the instrumented classes list ( for example,
> instrument
> > all the classes instead of only public API classes).
> >
> > I'd like to share our customized EMMA and the experience of using it. It
> is
> > highly welcomed to share your idea to automatize this process, or
> simplify
> > the it, or make the it more stably.
>
> It would be great if you would share it, then perhaps other people will
> see ways to improve creating the booting classes list -- and it would be
> interesting to get reports on the current BTI test coverage.
>
> Regards,
> Tim
>



-- 
Robert Hu
China Software Development Lab, IBM

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message