harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nathan Beyer" <nbe...@gmail.com>
Subject Re: [classlib] Code coverage
Date Tue, 21 Aug 2007 03:29:08 GMT
I've never used EMMA, so I have no idea. What about Cobertura?

On 8/20/07, Rui Hu <roberthurui@gmail.com> wrote:
> I think, it is not easy to automatically generate the coverage report of our
> trunk. :-(
>
> Because Harmony is not a Java application, it's the JDK, so the original
> EMMA can not be use to collect the coverage data of Harmony. We've modified
> the source code of EMMA, and realize the coverage collecting for core-java
> APIs. In this way, we must notice that:
> 1. The coverage of the classes in luni-kernel.jar can not be collected. It's
> not a big problem, because there're not many classes in that jar and the
> coverage of those classes is considered to be very high (they're usually
> used kernel classes).
> 2. The customized EMMA needs extra list of booting classes to run, the class
> list can not be too big or too small. If it's too big, the EMMA may not
> work. If it's too small, the JDK can not initialize.
> 3. The coverage of the classes in the list mentioned in #2 will be always
> 100%, (only the class coverage will be 100%, the method/block coverage will
> be correct), it's reasonable for the REAL booting classes, but incorrect for
> the unnecessary classes. So, this is another reason why we need not a
> too-big list.
>
> The problem is from the booting classes list mentioned in #2. Ideally, this
> list includes the classes which is used during initialization of running a
> test suite. Unfortunately, we do not have method to get the list
> automatically, and it related with the revision of Harmony build, the
> classes we want to collect coverage data for, and the test suite we want to
> run.
> So we now must try to get the list from cutting down the big-enough list
> (usually the all-instrumented-classes-list) every time. If the test crash,
> we must modify the booting classes list or remove some "problem test cases".
>
>
> The only shortcut is use the current successful list as the start point, but
> it can not ensure the test will run successfully, especially we change the
> test suite or expand the instrumented classes list ( for example, instrument
> all the classes instead of only public API classes).
>
> I'd like to share our customized EMMA and the experience of using it. It is
> highly welcomed to share your idea to automatize this process, or simplify
> the it, or make the it more stably.
>
> Thanks in advance !
>
> 2007/8/20, Nathan Beyer <nbeyer@gmail.com>:
> >
> > A patch for the build scripts would be best, as the coverage could
> > then be generated at any time by anyone.
> >
> > -Nathan
> >
> > On 8/19/07, Rui Hu <roberthurui@gmail.com> wrote:
> > > I've just generated a coverage report on Harmony trunk5 and branch 6 by
> > EMMA
> > > recently. Only public API classes are considered. The EMMA I used was
> > > customized to collect the coverage of core java api.
> > > The general result is:
> > > class% --- 68% (1641/2421)
> > > method% --- 60% (16303/27305)
> > > block% --- 58% (330724/572138)
> > >
> > > The following modules are poor covered , even "zero covered":
> > > concurrent, CORBA(ORB), XML, RMI, imageio
> > >
> > > coverage of awt/swing is not very high, we can improve the whole
> > coverage
> > > rapidly by improve these two big module.
> > >
> > > BTW,  I need a place to publish the report (folder of many HTMLs), any
> > > advice? Thanks a lot!
> > >
> > >
> > >
> > > 2007/8/19, Imran Ghory <imranghory@gmail.com>:
> > > >
> > > > What the current status with regards to code coverage of the junit
> > tests ?
> > > > -ve seen the wiki page but it seems a bit out of date (last october).
> > I've
> > > > tried getting emma to work myself based upon a few of the mailing list
> > > > posts
> > > > about it without much luck.
> > > >
> > > > Has anyone managed to get an emma instrumented build recently ?
> > > >
> > > > Imran
> > > >
> > >
> > >
> > >
> > > --
> > > Robert Hu
> > > China Software Development Lab, IBM
> > >
> >
>
>
>
> --
> Robert Hu
> China Software Development Lab, IBM
>

Mime
View raw message