harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Leo Li" <liyilei1...@gmail.com>
Subject Re: [classlib] Code coverage
Date Fri, 24 Aug 2007 09:48:51 GMT
On 8/24/07, Tim Ellison <t.p.ellison@gmail.com> wrote:
>
> Leo Li wrote:
> > On 8/23/07, Rui Hu <roberthurui@gmail.com> wrote:
> >> I've shared the customized emma tool and its usage in [1].
> >>
> >> We can also find the reason why we need customize emma and how to do
> it.
> >> The
> >> modified source code is also provided.
> >>
> >> Any comment is highly welcomed.
> >>
> >> [1]: http://wiki.apache.org/harmony/coverageEMMA
> >
> >
> >    The existence of kernel list is, as Robert says, because that Emma
> itself
> > depends on java core classes in its static initialization but
> > unfortunately in order to record testcoverage of these java core
> classes, in
> > their static initialization, emma will be referenced in order to
> register
> > some data structure for these java core classes. Thus Emma'
> initialization
> > depends on java core classes and java core classes' initialization
> depends
> > on Emma and VM detects a circular dependency and throw out
> ClassDefNotFound
> > exception.
> >    In out current solution, core classes in the kernel list will not
> > register themselves to emma during instrumentation. But during
> startup,  we
> > will explicitly load these core classes and Emma  and then manually
> register
> > core classes afterwards.
> >    But what puzzles me is that the kernel list, which is the
> dependencies of
> > Emma(The classes will be loaded when loading Emma.), is so unstable and
> is a
> > lot larger than my estimation. I once used JVMTI to record all the
> classes
> > loaded when loading Emma,  and the kernel list generated by this way is
> much
> > smaller and does not work.
> >   So is there a more precise way to calculate which class will be
> referenced
> > while loading Emma(The com.vladium.emma.rt.RT class)? I believe the
> > dependencies of this class should be more stable than we have met.
>
> How about "java -verbose:class ..."?  That will tell you which classes
> are loaded.  But you should have been able to get that info via JVMTI so
> maybe I misunderstand.



   Thanks, Tim. I will try it.:)

Regards,
> Tim
>
> > 2007/8/22, Rui Hu <roberthurui@gmail.com>:
> >>> I've shared the coverage report in HARMONY-4665. Anyone can download
> it
> >> to
> >>> see the coverage details. Thanks.
> >>> The customized EMMA and usage will be shared later.
> >>>
> >>>
> >>> 2007/8/21, Tim Ellison < t.p.ellison@gmail.com>:
> >>>> Rui Hu wrote:
> >>>>> I think, it is not easy to automatically generate the coverage
> >> report
> >>>> of our
> >>>>> trunk. :-(
> >>>>>
> >>>>> Because Harmony is not a Java application, it's the JDK, so the
> >>>> original
> >>>>> EMMA can not be use to collect the coverage data of Harmony. We've
> >>>> modified
> >>>>> the source code of EMMA, and realize the coverage collecting for
> >>>> core-java
> >>>>> APIs. In this way, we must notice that:
> >>>>> 1. The coverage of the classes in luni-kernel.jar can not be
> >>>> collected. It's
> >>>>> not a big problem, because there're not many classes in that jar
and
> >>>> the
> >>>>> coverage of those classes is considered to be very high (they're
> >>>> usually
> >>>>> used kernel classes).
> >>>>> 2. The customized EMMA needs extra list of booting classes to run,
> >> the
> >>>> class
> >>>>> list can not be too big or too small. If it's too big, the EMMA
may
> >>>> not
> >>>>> work. If it's too small, the JDK can not initialize.
> >>>>> 3. The coverage of the classes in the list mentioned in #2 will
be
> >>>> always
> >>>>> 100%, (only the class coverage will be 100%, the method/block
> >> coverage
> >>>> will
> >>>>> be correct), it's reasonable for the REAL booting classes, but
> >>>> incorrect for
> >>>>> the unnecessary classes. So, this is another reason why we need
not
> >> a
> >>>>> too-big list.
> >>>>>
> >>>>> The problem is from the booting classes list mentioned in #2.
> >> Ideally,
> >>>> this
> >>>>> list includes the classes which is used during initialization of
> >>>> running a
> >>>>> test suite. Unfortunately, we do not have method to get the list
> >>>>> automatically, and it related with the revision of Harmony build,
> >> the
> >>>>> classes we want to collect coverage data for, and the test suite
we
> >>>> want to
> >>>>> run.
> >>>>> So we now must try to get the list from cutting down the big-enough
> >>>> list
> >>>>> (usually the all-instrumented-classes-list) every time. If the test
> >>>> crash,
> >>>>> we must modify the booting classes list or remove some "problem
test
> >>>> cases".
> >>>>> The only shortcut is use the current successful list as the start
> >>>> point, but
> >>>>> it can not ensure the test will run successfully, especially we
> >> change
> >>>> the
> >>>>> test suite or expand the instrumented classes list ( for example,
> >>>> instrument
> >>>>> all the classes instead of only public API classes).
> >>>>>
> >>>>> I'd like to share our customized EMMA and the experience of using
> >> it.
> >>>> It is
> >>>>> highly welcomed to share your idea to automatize this process, or
> >>>> simplify
> >>>>> the it, or make the it more stably.
> >>>> It would be great if you would share it, then perhaps other people
> >> will
> >>>> see ways to improve creating the booting classes list -- and it would
> >> be
> >>>> interesting to get reports on the current BTI test coverage.
> >>>>
> >>>> Regards,
> >>>> Tim
> >>>>
> >>>
> >>>
> >>> --
> >>> Robert Hu
> >>> China Software Development Lab, IBM
> >>>
> >>
> >>
> >> --
> >> Robert Hu
> >> China Software Development Lab, IBM
> >>
> >
> >
> >
>



-- 
Leo Li
China Software Development Lab, IBM

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message