harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tim Ellison <t.p.elli...@gmail.com>
Subject Re: [classlib] Code coverage
Date Tue, 21 Aug 2007 10:12:26 GMT
Rui Hu wrote:
> I think, it is not easy to automatically generate the coverage report of our
> trunk. :-(
> 
> Because Harmony is not a Java application, it's the JDK, so the original
> EMMA can not be use to collect the coverage data of Harmony. We've modified
> the source code of EMMA, and realize the coverage collecting for core-java
> APIs. In this way, we must notice that:
> 1. The coverage of the classes in luni-kernel.jar can not be collected. It's
> not a big problem, because there're not many classes in that jar and the
> coverage of those classes is considered to be very high (they're usually
> used kernel classes).
> 2. The customized EMMA needs extra list of booting classes to run, the class
> list can not be too big or too small. If it's too big, the EMMA may not
> work. If it's too small, the JDK can not initialize.
> 3. The coverage of the classes in the list mentioned in #2 will be always
> 100%, (only the class coverage will be 100%, the method/block coverage will
> be correct), it's reasonable for the REAL booting classes, but incorrect for
> the unnecessary classes. So, this is another reason why we need not a
> too-big list.
> 
> The problem is from the booting classes list mentioned in #2. Ideally, this
> list includes the classes which is used during initialization of running a
> test suite. Unfortunately, we do not have method to get the list
> automatically, and it related with the revision of Harmony build, the
> classes we want to collect coverage data for, and the test suite we want to
> run.
> So we now must try to get the list from cutting down the big-enough list
> (usually the all-instrumented-classes-list) every time. If the test crash,
> we must modify the booting classes list or remove some "problem test cases".
> 
> The only shortcut is use the current successful list as the start point, but
> it can not ensure the test will run successfully, especially we change the
> test suite or expand the instrumented classes list ( for example, instrument
> all the classes instead of only public API classes).
> 
> I'd like to share our customized EMMA and the experience of using it. It is
> highly welcomed to share your idea to automatize this process, or simplify
> the it, or make the it more stably.

It would be great if you would share it, then perhaps other people will
see ways to improve creating the booting classes list -- and it would be
interesting to get reports on the current BTI test coverage.

Regards,
Tim

Mime
View raw message