harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rui Hu" <roberthu...@gmail.com>
Subject Re: [classlib] Code coverage
Date Mon, 20 Aug 2007 09:15:07 GMT
I think, it is not easy to automatically generate the coverage report of our
trunk. :-(

Because Harmony is not a Java application, it's the JDK, so the original
EMMA can not be use to collect the coverage data of Harmony. We've modified
the source code of EMMA, and realize the coverage collecting for core-java
APIs. In this way, we must notice that:
1. The coverage of the classes in luni-kernel.jar can not be collected. It's
not a big problem, because there're not many classes in that jar and the
coverage of those classes is considered to be very high (they're usually
used kernel classes).
2. The customized EMMA needs extra list of booting classes to run, the class
list can not be too big or too small. If it's too big, the EMMA may not
work. If it's too small, the JDK can not initialize.
3. The coverage of the classes in the list mentioned in #2 will be always
100%, (only the class coverage will be 100%, the method/block coverage will
be correct), it's reasonable for the REAL booting classes, but incorrect for
the unnecessary classes. So, this is another reason why we need not a
too-big list.

The problem is from the booting classes list mentioned in #2. Ideally, this
list includes the classes which is used during initialization of running a
test suite. Unfortunately, we do not have method to get the list
automatically, and it related with the revision of Harmony build, the
classes we want to collect coverage data for, and the test suite we want to
run.
So we now must try to get the list from cutting down the big-enough list
(usually the all-instrumented-classes-list) every time. If the test crash,
we must modify the booting classes list or remove some "problem test cases".


The only shortcut is use the current successful list as the start point, but
it can not ensure the test will run successfully, especially we change the
test suite or expand the instrumented classes list ( for example, instrument
all the classes instead of only public API classes).

I'd like to share our customized EMMA and the experience of using it. It is
highly welcomed to share your idea to automatize this process, or simplify
the it, or make the it more stably.

Thanks in advance !

2007/8/20, Nathan Beyer <nbeyer@gmail.com>:
>
> A patch for the build scripts would be best, as the coverage could
> then be generated at any time by anyone.
>
> -Nathan
>
> On 8/19/07, Rui Hu <roberthurui@gmail.com> wrote:
> > I've just generated a coverage report on Harmony trunk5 and branch 6 by
> EMMA
> > recently. Only public API classes are considered. The EMMA I used was
> > customized to collect the coverage of core java api.
> > The general result is:
> > class% --- 68% (1641/2421)
> > method% --- 60% (16303/27305)
> > block% --- 58% (330724/572138)
> >
> > The following modules are poor covered , even "zero covered":
> > concurrent, CORBA(ORB), XML, RMI, imageio
> >
> > coverage of awt/swing is not very high, we can improve the whole
> coverage
> > rapidly by improve these two big module.
> >
> > BTW,  I need a place to publish the report (folder of many HTMLs), any
> > advice? Thanks a lot!
> >
> >
> >
> > 2007/8/19, Imran Ghory <imranghory@gmail.com>:
> > >
> > > What the current status with regards to code coverage of the junit
> tests ?
> > > -ve seen the wiki page but it seems a bit out of date (last october).
> I've
> > > tried getting emma to work myself based upon a few of the mailing list
> > > posts
> > > about it without much luck.
> > >
> > > Has anyone managed to get an emma instrumented build recently ?
> > >
> > > Imran
> > >
> >
> >
> >
> > --
> > Robert Hu
> > China Software Development Lab, IBM
> >
>



-- 
Robert Hu
China Software Development Lab, IBM

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message