# harmony-dev mailing list archives

##### Site index · List index
Message view
Top
Subject Re: [classlib] [testing] Coverage (was Re: 37% of total test execution time is spent in a single test)
Date Tue, 13 Jun 2006 11:50:12 GMT
```> One more thing I've missed: what is the exclude list you've finally ended
with?
I'm going to publish it ASAP.

> > P.S. I think java.awt does not currently belong to beans module,
> > though who knows what the future may bring us :)

In fact, coverage information is gathered for each jar, that, to some extend
corresponds to modules content.
Some awt classes are in beans.jar, that is why they appeared in coverage
statistics for beans module.

Thanks,

On 6/13/06, Mikhail Loenko <mloenko@gmail.com> wrote:

> Cool! Sorry, I've missed it
>
> One more thing I've missed: what is the exclude list you've finally ended
> with?
>
> Thanks,
> Mikhail
>
> > The detailed info available at  http://viv.byethost15.com/ (this address
> > also pointed on wiki-page).
> > Do you need additional information ?
> >
> > Thanks,
> >
> >
> > On 6/13/06, Mikhail Loenko < mloenko@gmail.com> wrote:
> > >
> > > Great!
> > >
> > > Is it possible to see uncovered classes/methods/lines?
> > >
> > > Thanks,
> > > Mikhail
> > >
> > > P.S. I think java.awt does not currently belong to beans module,
> > > though who knows what the future may bring us :)
> > >
> > > > Latest Harmony API source coverage by Harmony API unit tests results
> I
> > > > stored at wiki page
> > > >
> > > > http://wiki.apache.org/harmony/Coverage_information
> > > >
> > > >
> > > >
> > > > I'm going to refresh it bi-weekly (seems, it is enough for
> coverage).
> > > >
> > > >
> > > >
> > > > >I think we have got a agreement on the test naming convention[1],
> but
> > > > >for sure, there have been many (legacy) test cases before this
> > > > >agreement, and I think the volunteer is highly welcome to provide
> patch
> > > > >for them.
> > > > >
> > > >
> > >
> >[1]http://incubator.apache.org/harmony/subcomponents/classlibrary/testing.html
> > > >
> > > > >
> > > >
> > > > If nobody objects I'm going to look through the unit tests to
> correct
> > > > package names according to the agreement (where needed).
> > > >
> > > >
> > > >
> > > >  Thanks,
> > > >
> > > >
> > > >
> > > >
> > > > On 6/8/06, Paulex Yang <paulex.yang@gmail.com > wrote:
> > > > >
> > > > > Vladimir,
> > > > >
> > > > > Vladimir Ivanov wrote:
> > > > > > Thanks Paulex!
> > > > > >
> > > > > > I did the same, but could not send results due to spam filter
J
> > > > > > Observations:
> > > > > >
> > > > > >   1. Coverage results look pretty much similar.
> > > > > >   2. Exclude list looks pretty much similar too, but, looks
like
> it
> > > > > >   depends on the way of data collection (I didn't run ant task
> and
> > > the
> > > > > > list is
> > > > > >   a little bit different).
> > > > > Great.
> > > > > >
> > > > > > In any case, I think, when we run harmony on another VM exclude
> list
> > > > > will
> > > > > > have to be updated.
> > > > > >
> > > > > >
> > > > > >
> > > > > > May be we can start publishing the coverage information on wiki
> > > pages
> > > > > and
> > > > > > provide some updates time to time (I can do it)?
> > > > > >
> > > > > +1,  and of course, you can only if  no one in the mailing list
> > > objects
> > > > > and you'll have my welcome, and I think it will be even greater if
>
> > > these
> > > > > reports can be generated regularly like what JAPI is doing:)
> > > > > >
> > > > > >
> > > > > > One note:
> > > > > >
> > > > > > I noticed that different unit tests have very different package
> > > names
> > > > > >
> > > > > > Now the directory with all built tests copied to one place looks
>
> > > like:
> > > > > I think we have got a agreement on the test naming convention[1],
> but
> > > > > for sure, there have been many (legacy) test cases before this
> > > > > agreement, and I think the volunteer is highly welcome to provide
> > > patch
> > > > > for them.
> > > > >
> > > > >
> > >
> [1]http://incubator.apache.org/harmony/subcomponents/classlibrary/testing.html
> > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > C:\coverage\tests\test>ls
> > > > > >
> > > > > > GZIPOutClose2.txt        api           config         javax
> > > > > > tests
> > > > > >
> > > > > > GZIPOutFinish.txt        api.injected  dazzle
> > > org            xml
> > > > >
> > > > > >
> > > > > > GZIPOutWrite.txt         binary        gif            prefs
> > > > > >
> > > > > > Inet6Address.golden.ser  bundles       impl
> serialization
> > > > > >
> > > > > > JDK2-3gabba.zip          com           impl.injected   test.txt
> > > > > >
> > > > > >
> > > > > >
> > > > > > I think, it would be good if tests had unified package names.
> > > > > >
> > > > > > Why? – so far, just common sense, just to have an order in
test
> > > suite
> > > > > >
> > > > > > Organization (if consider all unit tests as solid test suite).
> > > > > >
> > > > > > Thanks,
> > > > > >  Vladimir
> > > > > >
> > > > > > For example, my exclude list for java.io is:
> > > > > > -java.io.BufferedInputStream ,
> > > > > > -java.io.BufferedOutputStream,
> > > > > > -java.io.File ,
> > > > > > -java.io.FileChannelFactory ,
> > > > > > -java.io.FileDescriptor,
> > > > > > -java.io.FileInputStream,
> > > > > > -java.io.FileOutputStream,
> > > > > > -java.io.FilterInputStream ,
> > > > > > -java.io.FilterOutputStream,
> > > > > > - java.io.InputStream,
> > > > > > -java.io.OutputStream,
> > > > > > -java.io.ObjectStreamField,
> > > > > > - java.io.PrintStream
> > > > > >
> > > > > >
> > > > > > On 6/6/06, Paulex Yang < paulex.yang@gmail.com > wrote:
> > > > > >>
> > > > > >> I've attach the scripts and excluded class lists to
> > > refer
> > > > > >> to https://issues.apache.org/jira/browse/HARMONY-564 .
Enjoy
> it:).
> > > > > >>
> > > > > >> Mark Hindess wrote:
> > > > > >> > On 2 June 2006 at 10:37, Paulex Yang < paulex.yang@gmail.com>
> > > wrote:
> > > > > >> >
> > > > > >> >> Mark,
> > > > > >> >>
> > > > > >> >> I'm glad that there is someone else has interest
on emma,
> I've
> > > > > >> tried it
> > > > > >> >> before. AFAIK, emma works by instrumentation, but
sometimes
> for
> > > > > >> classes
> > > > > >> >> in bootclasspath, the instrumentation cannot work,
there are
> two
> > > > > >> cases:
> > > > > >>
> > > > > >> >> 1. Some instrumented classes cannot be loaded by
VM.
> > > > > >> >> 2. Some classes cannot be instrumented
> > > > > >> >>
> > > > > >> >> I have tried to look more inside to find some way
to work
> around
> > > > > >> but I
> > > > > >> >> haven't got enough time yet.
> > > > > >> >>
> > > > > >> >> Specifically for nio-channel module, I had a list
for these
> two
> > > > > cases
> > > > > >> (I
> > > > > >> >> believe the data is a little outdated and should
be
> reevaluated)
> > > > > >> >> case 1.
> > > > > >> >> BaseByteBuffer.class
> > > > > >> >> Buffer.class
> > > > > >> >> BufferFactory.class
> > > > > >> >> ByteBuffer.class
> > > > > >> >> CharArrayBuffer.class
> > > > > >> >> CharBuffer.class
> > > > > >> >> HeapByteBuffer.class
> > > > > >> >> ReadWriteCharArrayBuffer.class
> > > > > >> >> ReadWriteHeapByteBuffer.class
> > > > > >> >> FileChannel.class
> > > > > >> >> AbstractInterruptibleChannel.class
> > > > > >> >> FileChannelImpl.class
> > > > > >> >> WriteOnlyFileChannel.class
> > > > > >> >> LockManager.class
> > > > > >> >> LockManager\$1.class
> > > > > >> >> ReadOnlyFileChannel.class
> > > > > >> >>
> > > > > >> >> case 2:
> > > > > >> >> ByteChannel.class
> > > > > >> >> Channel.class
> > > > > >> >> GatheringByteChannel.class
> > > > > >> >> InterruptibleChannel.class
> > > > > >> >> WritableByteChannel.class
> > > > > >> >>
> > > > > >> >> And I have got some ant script and more excluded
list for
> emma,
> > > if
> > > > > >> >> anyone has interests, I can upload it to JIRA.
> > > > > >> >>
> > > > > >> >
> > > > > >> > Yes!
> > > > > >> >
> > > > > >> > -Mark.
> > > > > >> >
> > > > > >> >
> > > > > >> >> Mark Hindess wrote:
> > > > > >> >>
> > > > > >> >>> Anyone tried using emma (emma.sf.net) to look
at test
> coverage
> > > > > >> for our
> > > > > >> >>> tests?  That might help ensure we don't trim
them down too
> > > much?
> > > > > >> >>>
> > > > > >> >>> I suspect we will soon reach a point where
there are
> > > significantly
> > > > > >> fewer
> > > > > >> >>> obvious opportunities for small contributions
- e.g.
> > > > > >> 1.5generification,
> > > > > >> >>> etc from looking at JAPI reports.  Having test
coverage
> reports
> > > > > >> might
> > > > > >> >>> help by highlighting areas for improvement.
> > > > > >> >>>
> > > > > >> >>> I had a quick try with:
> > > > > >> >>>
> > > > > >> >>>   ant -f make/build.xml
> > > > > >> >>>   ant -f modules/nio/make/build.xml test
> > > > > >> >>>   java -cp emma.jar emma instr -m overwrite
-cp
> > > > > >> deploy/jdk/jre/lib/boot/nio
> > > > > >> >>>
> > > > > >> >> .jar
> > > > > >> >>
> > > > > >> >>> and then:
> > > > > >> >>>
> > > > > >> >>>   java -Xbootclasspath/p: emma.jar \
> > > > > >> >>>        -cp build/tests:modules/nio/bin/test:
junit.jar:. \
> > > > > >> >>>        junit.textui.TestRunner \
> > > > > >> >>>
> > > org.apache.harmony.tests.java.nio.channels.SocketChannelTest
> > > > > >> >>>
> > > > > >> >>> (Running all the tests is a little tricky because
test
> depends
> > > on
> > > > > >> build,
> > > > > >> >>> which depends on clean which wipes out the
instr'd jar. ;-(
> )
> > > > > >> >>>
> > > > > >> >>> Sadly it dies horribly with the stack trace
below.  I'll
> take
> > > > > >> another
> > > > > >> >>> look when I have a little more time, but I
thought I'd
> mention
> > > it
> > > > > in
> > > > > >> >>> case anyone else had any thoughts on the subject.
> > > > > >> >>>
> > > > > >> >>> Regards,
> > > > > >> >>>
> > > > > >> >>> Mark.
> > > > > >> >>>
> > > > > >> >>> Exception in thread "main"
> > > java/lang/ExceptionInInitializerError
> > > > > >> >>>         at java/lang/J9VMInternals.initialize
> > > > > >> (J9VMInternals.java:203)
> > > > > >> >>>         at java/nio/Buffer.\$VRi ( Buffer.java:4294967295)
> > > > > >> >>>         at java/nio/Buffer.<clinit> (
Buffer.java
> :4294967295)
> > > > > >> >>>         at java/lang/J9VMInternals.initializeImpl
(Native
> > > Method)
> > > > > >> >>>         at java/lang/J9VMInternals.initialize
(
> > > J9VMInternals.java
> > > > > >> :185)
> > > > > >> >>>         at java/lang/J9VMInternals.initialize
> > > > > >> (J9VMInternals.java:154)
> > > > > >> >>>         at java/lang/String.getBytes (String.java:783)
> > > > > >> >>>         at org/apache/harmony/luni/util/Util.getBytes
(
> > > > > >> Util.java:54)
> > > > > >> (ClassLoader.java
> > > > > >> :998)
> (
> > > > > >> ClassLoader.ja
> > > > > >> >>>
> > > > > >> >> va:970)
> > > > > >> >>
> > > > > >> >>>         at java/lang/System.loadLibrary (System.java:706)
>
> > > > > (Unknown
> > > > > >> Source
> > > > > >> >>>
> > > > > >> >> )
> > > > > >> >>
> > > > > >> >>>         at
> com/ibm/icu4jni/converters/NativeConverter.<clinit>
> > > > > >> (Unknown Sou
> > > > > >> >>>
> > > > > >> >> rce)
> > > > > >> >>
> > > > > >> >>>         at java/lang/J9VMInternals.initializeImpl
(Native
> > > Method)
> > > > > >> >>>         at java/lang/J9VMInternals.initialize
> > > > > >> ( J9VMInternals.java:185)
> > > > > >> >>>         at
> > > > > com/ibm/icu4jni/charset/CharsetProviderICU.charsetForName
> > > > > >> (Unkno
> > > > > >> >>>
> > > > > >> >> wn Source)
> > > > > >> >>
> > > > > >> >>>         at java/nio/charset/Charset.forNameInternal
> > > > > >> (Charset.java:450)
> > > > > >> >>>         at java/nio/charset/Charset.forName
( Charset.java
> :513)
> > > > > >> >>>         at java/lang/String.defaultCharset
( String.java
> :688)
> > > > > >> >>>         at java/lang/String.<init> (
String.java :252)
> > > > > >> >>>         at org/apache/harmony/luni/util/Util.toString
(
> > > Util.java
> > > > > :74)
> > > > > >> >>>         at java/lang/System.getPropertyList
(Native Method)
>
> > > > > >> >>>         at java/lang/System.ensureProperties
( System.java
> :496)
> > > > > >> >>>         at java/lang/System.<clinit>
(System.java:55)
> > > > > >> >>>         at java/lang/J9VMInternals.initializeImpl
(Native
> > > Method)
> > > > > >> >>>         at java/lang/J9VMInternals.initialize
> > > > > >> (J9VMInternals.java:185)
(
> > > > > >> ClassLoader.java:6
> > > > > >> >>>
> > > > > >> >> 5)
> > > > > >> >>
> > > > > >> >>> java/lang/NullPointerException
> > > > > >> >>>         at java/lang/Throwable.printStackTrace
(
> Throwable.java
> > > > > :231)
> > > > > >> >>>         at
> > > > > >> (RTExitH
> > > > > >> >>>
> > > > > >> >> ook.java:63)
> > > > > >> >>
> > > > > >> >>>         at com/vladium/emma/rt/RT.reset ( RT.java:92)
> > > > > >> >>>         at com/vladium/emma/rt/RT.<clinit>
( RT.java:238)
> > > > > >> >>>         at java/lang/J9VMInternals.initializeImpl
(Native
> > > Method)
> > > > > >> >>>         at java/lang/J9VMInternals.initialize
(
> > > J9VMInternals.java
> > > > > >> :185)
> > > > > >> >>>         at java/nio/Buffer.\$VRi ( Buffer.java:4294967295)
> > > > > >> >>>         at java/nio/Buffer.<clinit> (
Buffer.java
> :4294967295)
> > > > > >> >>>         at java/lang/J9VMInternals.initializeImpl
(Native
> > > Method)
> > > > > >> >>>         at java/lang/J9VMInternals.initialize
> > > > > >> (J9VMInternals.java:185)
> > > > > >> >>>         at java/lang/J9VMInternals.initialize
> > > > > >> (J9VMInternals.java:154)
> > > > > >> >>>         at java/lang/String.getBytes ( String.java
:783)
> > > > > >> >>>         at org/apache/harmony/luni/util/Util.getBytes
(
> > > Util.java
> > > > > :54)
> > > > > >> (ClassLoader.java
> > > > > >> :998)
> (
> > > > > >> ClassLoader.ja
> > > > > >> >>>
> > > > > >> >> va:970)
> > > > > >> >>
> > > > > >> >>>         at java/lang/System.loadLibrary ( System.java:706)
> > > > > (Unknown
> > > > > >> Source
> > > > > >> >>>
> > > > > >> >> )
> > > > > >> >>
> > > > > >> >>>         at
> com/ibm/icu4jni/converters/NativeConverter.<clinit>
> > > > > >> (Unknown Sou
> > > > > >> >>>
> > > > > >> >> rce)
> > > > > >> >>
> > > > > >> >>>         at java/lang/J9VMInternals.initializeImpl
(Native
> > > Method)
> > > > > >> >>>         at java/lang/J9VMInternals.initialize
> > > > > >> (J9VMInternals.java:185)
> > > > > >> >>>         at
> > > > > com/ibm/icu4jni/charset/CharsetProviderICU.charsetForName
> > > > > >> (Unkno
> > > > > >> >>>
> > > > > >> >> wn Source)
> > > > > >> >>
> > > > > >> >>>         at java/nio/charset/Charset.forNameInternal
> > > > > >> (Charset.java:450)
> > > > > >> >>>         at java/nio/charset/Charset.forName
( Charset.java
> :513)
> > > > > >> >>>         at java/lang/String.defaultCharset
( String.java
> :688)
> > > > > >> >>>         at java/lang/String.<init> (String.java
:252)
> > > > > >> >>>         at org/apache/harmony/luni/util/Util.toString
(
> > > Util.java
> > > > > :74)
> > > > > >> >>>         at java/lang/System.getPropertyList
(Native Method)
>
> > > > > >> >>>         at java/lang/System.ensureProperties
(System.java
> :496)
> > > > > >> >>>         at java/lang/System.<clinit>
(System.java:55)
> > > > > >> >>>         at java/lang/J9VMInternals.initializeImpl
(Native
> > > Method)
> > > > > >> >>>         at java/lang/J9VMInternals.initialize
> > > > > >> (J9VMInternals.java:185)
(
> > > > > >> ClassLoader.java:6
> > > > > >> >>>
> > > > > >> >> 5)
> > > > > >> >>
:120)
> > > > > >> >>> JVMJ9VM015W Initialization error for library
> jclclear_23(14):
> > > > > >> JVMJ9VM009E J
> > > > > >> >>>
> > > > > >> >> 9VMDllMain failed
> > > > > >> >>
> > > > > >> >>> HMYEXEL062E Internal VM error: Failed to create
Java VM
> > > > > >> >>> FAILED.
> > > > > >> >>>
> > > > > >> >>>
> > > > > >> >>> On 1 June 2006 at 22:26, "Mikhail Loenko" <
> mloenko@gmail.com >
> > > > > >> wrote:
> > > > > >> >>>
> > > > > >> >>>
> > > > > >> >>>> 2006/6/1, Geir Magnusson Jr < geir@pobox.com>:
> > > > > >> >>>>
> > > > > >> >>>>
> > > > > >> >>>>> Mikhail Loenko wrote:
> > > > > >> >>>>>
> > > > > >> >>>>>
> > > > > >> >>>>>> 2006/6/1, Geir Magnusson Jr <
geir@pobox.com >:
> > > > > >> >>>>>>
> > > > > >> >>>>>>
> > > > > >> >>>>>>> I don't quite grok what you
mean - could you post what
> you
> > > are
> > > > > >> >>>>>>> suggesting in code here on
-dev?
> > > > > >> >>>>>>>
> > > > > >> >>>>>>>
> > > > > >> >>>>>> Index:
> > > > > >>
> > > > > >> >>>>>>
> > > > > >> >> v
> > > > > >> >>
> > > > > >> >>>>>>
> > > > > >> >>>>>>
> > > > > >> >>>> a
> > > > > >> >>>>
> > > > > >> >>>>
> > > > > >> >>>>>>
> > > > > >>
> ===================================================================
> > > > > >> >>>>>> ---
> > > > > >> >>>>>>
> > > > > >>
> > > > > >> >>>>>> (revision
> > > > > >> >>>>>> 410480)
> > > > > >> >>>>>> +++
> > > > > >> >>>>>>
> > > > > >>
> > > > > >>
> > > > > >> >>>>>> (working
> > > > > >> >>>>>> copy)
> > > > > >> >>>>>> @@ -785,7 +785,7 @@
> > > > > >> >>>>>>                 "FFFF:FFFF:FFFF:FFFF:FFFF:FFFF:
> > > 255.255.255.255
> > > > > ",
> > > > > >> >>>>>>                 "0:0:0:0:0:0:0:0",
"0:0:0:0:0:0: 0.0.0.0"
> };
> > > > > >> >>>>>>
> > > > > >> >>>>>> -        String invalidIPAddresses[]
= { ":",
> "FFFF:FFFF",
> > > > > "1:1",
> > > > > >> >>>>>> "::1.2.3.444" };
> > > > > >> >>>>>> +        String invalidIPAddresses[]
= { "FFFF:FFFF" };
> > > > > >> >>>>>>
> > > > > >> >>>>>>         for (int i = 0; i <
> i++) {
> > > > > >> >>>>>>
> > > > > >> >>>>>>
> > > > > >> >>>>>>
> > > > > >> >>>>>>
> > > > > >> >>>>> Thx - why do you want to remove testing
the
> others?  Because
> > > you
> > > > > >> believe
> > > > > >> >>>>> that it is in effect testing the resolver
in the OS since
> > > each
> > > > > >> returns
> > > > > >> >>>>> the same answer,and thus it's testing
the same thing over
> and
> > > > > >> over?
> > > > > >> >>>>>
> > > > > >> >>>>>
> > > > > >> >>>> I believe so, but I'd like to hear confirmation
from code
> > > authors
> > > > > >> >>>>
> > > > > >> >>>> Thanks,
> > > > > >> >>>> Mikhail
> > > > > >> >>>>
> > > > > >> >>>>
> > > > > >> >>>
> > > > > >> >>>
> > > > > >>
> > > ---------------------------------------------------------------------
> http://incubator.apache.org/harmony/mailing.html
> > > > > >> >>> To unsubscribe, e-mail:
> > > harmony-dev-unsubscribe@incubator.apache.org
> > > > >
> > > > > >> >>> For additional commands, e-mail:
> > > > > >> harmony-dev-help@incubator.apache.org
> > > > > >> >>>
> > > > > >> >>>
> > > > > >> >>>
> > > > > >> >>>
> > > > > >> >> --
> > > > > >> >> Paulex Yang
> > > > > >> >> China Software Development Lab
> > > > > >> >> IBM
> > > > > >> >>
> > > > > >> >>
> > > > > >> >>
> > > > > >> >>
> > > > >
> ---------------------------------------------------------------------
> http://incubator.apache.org/harmony/mailing.html
> > > > > >> >> To unsubscribe, e-mail:
> > > harmony-dev-unsubscribe@incubator.apache.org
> > > > > >> >> For additional commands, e-mail:
> > > > > >> harmony-dev-help@incubator.apache.org
> > > > > >> >>
> > > > > >> >
> > > > > >> >
> > > > > >> >
> > > > > >> >
> > > ---------------------------------------------------------------------
> > > > >
> http://incubator.apache.org/harmony/mailing.html
> > > > > >> > To unsubscribe, e-mail:
> > > harmony-dev-unsubscribe@incubator.apache.org
> > > > > >> > For additional commands, e-mail:
> > > > > harmony-dev-help@incubator.apache.org
> > > > > >> >
> > > > > >> >
> > > > > >> >
> > > > > >>
> > > > > >>
> > > > > >> --
> > > > > >> Paulex Yang
> > > > > >> China Software Development Lab
> > > > > >> IBM
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >>
> > > ---------------------------------------------------------------------
> > > > > >> To unsubscribe, e-mail:
> > > harmony-dev-unsubscribe@incubator.apache.org
> > > > > >> For additional commands, e-mail:
> > > harmony-dev-help@incubator.apache.org
> > > > > >>
> > > > > >>
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Paulex Yang
> > > > > China Software Development Lab
> > > > > IBM
> > > > >
> > > > >
> > > > >
> > > > >
> ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail:
> harmony-dev-unsubscribe@incubator.apache.org
> > > > > For additional commands, e-mail:
> harmony-dev-help@incubator.apache.org
> > > > >
> > > > >
> > > >
> > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > >
> > >
> >
> >
>
> ---------------------------------------------------------------------