harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rui Hu" <roberthu...@gmail.com>
Subject Re: [classlib] Code coverage
Date Thu, 23 Aug 2007 10:24:08 GMT
I've shared the customized emma tool and its usage in [1].

We can also find the reason why we need customize emma and how to do it. The
modified source code is also provided.

Any comment is highly welcomed.

[1]: http://wiki.apache.org/harmony/coverageEMMA

2007/8/22, Rui Hu <roberthurui@gmail.com>:
>
> I've shared the coverage report in HARMONY-4665. Anyone can download it to
> see the coverage details. Thanks.
> The customized EMMA and usage will be shared later.
>
>
> 2007/8/21, Tim Ellison < t.p.ellison@gmail.com>:
> >
> > Rui Hu wrote:
> > > I think, it is not easy to automatically generate the coverage report
> > of our
> > > trunk. :-(
> > >
> > > Because Harmony is not a Java application, it's the JDK, so the
> > original
> > > EMMA can not be use to collect the coverage data of Harmony. We've
> > modified
> > > the source code of EMMA, and realize the coverage collecting for
> > core-java
> > > APIs. In this way, we must notice that:
> > > 1. The coverage of the classes in luni-kernel.jar can not be
> > collected. It's
> > > not a big problem, because there're not many classes in that jar and
> > the
> > > coverage of those classes is considered to be very high (they're
> > usually
> > > used kernel classes).
> > > 2. The customized EMMA needs extra list of booting classes to run, the
> > class
> > > list can not be too big or too small. If it's too big, the EMMA may
> > not
> > > work. If it's too small, the JDK can not initialize.
> > > 3. The coverage of the classes in the list mentioned in #2 will be
> > always
> > > 100%, (only the class coverage will be 100%, the method/block coverage
> > will
> > > be correct), it's reasonable for the REAL booting classes, but
> > incorrect for
> > > the unnecessary classes. So, this is another reason why we need not a
> > > too-big list.
> > >
> > > The problem is from the booting classes list mentioned in #2. Ideally,
> > this
> > > list includes the classes which is used during initialization of
> > running a
> > > test suite. Unfortunately, we do not have method to get the list
> > > automatically, and it related with the revision of Harmony build, the
> > > classes we want to collect coverage data for, and the test suite we
> > want to
> > > run.
> > > So we now must try to get the list from cutting down the big-enough
> > list
> > > (usually the all-instrumented-classes-list) every time. If the test
> > crash,
> > > we must modify the booting classes list or remove some "problem test
> > cases".
> > >
> > > The only shortcut is use the current successful list as the start
> > point, but
> > > it can not ensure the test will run successfully, especially we change
> > the
> > > test suite or expand the instrumented classes list ( for example,
> > instrument
> > > all the classes instead of only public API classes).
> > >
> > > I'd like to share our customized EMMA and the experience of using it.
> > It is
> > > highly welcomed to share your idea to automatize this process, or
> > simplify
> > > the it, or make the it more stably.
> >
> > It would be great if you would share it, then perhaps other people will
> > see ways to improve creating the booting classes list -- and it would be
> > interesting to get reports on the current BTI test coverage.
> >
> > Regards,
> > Tim
> >
>
>
>
> --
> Robert Hu
> China Software Development Lab, IBM
>



-- 
Robert Hu
China Software Development Lab, IBM

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message