harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vladimir Ivanov" <ivavladi...@gmail.com>
Subject Re: [classlib] [testing] Coverage (was Re: 37% of total test execution time is spent in a single test)
Date Tue, 13 Jun 2006 05:36:24 GMT
Latest Harmony API source coverage by Harmony API unit tests results I
stored at wiki page

http://wiki.apache.org/harmony/Coverage_information



I'm going to refresh it bi-weekly (seems, it is enough for coverage).



>I think we have got a agreement on the test naming convention[1], but
>for sure, there have been many (legacy) test cases before this
>agreement, and I think the volunteer is highly welcome to provide patch
>for them.
>
>[1]http://incubator.apache.org/harmony/subcomponents/classlibrary/testing.html

>

If nobody objects I'm going to look through the unit tests to correct
package names according to the agreement (where needed).



 Thanks,

   Vladimir



On 6/8/06, Paulex Yang <paulex.yang@gmail.com > wrote:
>
> Vladimir,
>
> Vladimir Ivanov wrote:
> > Thanks Paulex!
> >
> > I did the same, but could not send results due to spam filter J
> > Observations:
> >
> >   1. Coverage results look pretty much similar.
> >   2. Exclude list looks pretty much similar too, but, looks like it
> >   depends on the way of data collection (I didn't run ant task and the
> > list is
> >   a little bit different).
> Great.
> >
> > In any case, I think, when we run harmony on another VM exclude list
> will
> > have to be updated.
> >
> >
> >
> > May be we can start publishing the coverage information on wiki pages
> and
> > provide some updates time to time (I can do it)?
> >
> +1,  and of course, you can only if  no one in the mailing list objects
> and you'll have my welcome, and I think it will be even greater if these
> reports can be generated regularly like what JAPI is doing:)
> >
> >
> > One note:
> >
> > I noticed that different unit tests have very different package names
> >
> > Now the directory with all built tests copied to one place looks like:
> I think we have got a agreement on the test naming convention[1], but
> for sure, there have been many (legacy) test cases before this
> agreement, and I think the volunteer is highly welcome to provide patch
> for them.
>
> [1]http://incubator.apache.org/harmony/subcomponents/classlibrary/testing.html
>
> >
> >
> >
> > C:\coverage\tests\test>ls
> >
> > GZIPOutClose2.txt        api           config         javax
> > tests
> >
> > GZIPOutFinish.txt        api.injected  dazzle         org            xml
>
> >
> > GZIPOutWrite.txt         binary        gif            prefs
> >
> > Inet6Address.golden.ser  bundles       impl           serialization
> >
> > JDK2-3gabba.zip          com           impl.injected   test.txt
> >
> >
> >
> > I think, it would be good if tests had unified package names.
> >
> > Why? – so far, just common sense, just to have an order in test suite
> >
> > Organization (if consider all unit tests as solid test suite).
> >
> > Thanks,
> >  Vladimir
> >
> > For example, my exclude list for java.io is:
> > -java.io.BufferedInputStream ,
> > -java.io.BufferedOutputStream,
> > -java.io.File ,
> > -java.io.FileChannelFactory,
> > -java.io.FileDescriptor,
> > -java.io.FileInputStream,
> > -java.io.FileOutputStream,
> > -java.io.FilterInputStream ,
> > -java.io.FilterOutputStream,
> > - java.io.InputStream,
> > -java.io.OutputStream,
> > -java.io.ObjectStreamField,
> > -java.io.PrintStream
> >
> >
> > On 6/6/06, Paulex Yang < paulex.yang@gmail.com > wrote:
> >>
> >> I've attach the scripts and excluded class lists to JIRA,  please refer
> >> to https://issues.apache.org/jira/browse/HARMONY-564 .  Enjoy it:).
> >>
> >> Mark Hindess wrote:
> >> > On 2 June 2006 at 10:37, Paulex Yang < paulex.yang@gmail.com> wrote:
> >> >
> >> >> Mark,
> >> >>
> >> >> I'm glad that there is someone else has interest on emma, I've
> >> tried it
> >> >> before. AFAIK, emma works by instrumentation, but sometimes for
> >> classes
> >> >> in bootclasspath, the instrumentation cannot work, there are two
> >> cases:
> >>
> >> >> 1. Some instrumented classes cannot be loaded by VM.
> >> >> 2. Some classes cannot be instrumented
> >> >>
> >> >> I have tried to look more inside to find some way to work around
> >> but I
> >> >> haven't got enough time yet.
> >> >>
> >> >> Specifically for nio-channel module, I had a list for these two
> cases
> >> (I
> >> >> believe the data is a little outdated and should be reevaluated)
> >> >> case 1.
> >> >> BaseByteBuffer.class
> >> >> Buffer.class
> >> >> BufferFactory.class
> >> >> ByteBuffer.class
> >> >> CharArrayBuffer.class
> >> >> CharBuffer.class
> >> >> HeapByteBuffer.class
> >> >> ReadWriteCharArrayBuffer.class
> >> >> ReadWriteHeapByteBuffer.class
> >> >> FileChannel.class
> >> >> AbstractInterruptibleChannel.class
> >> >> FileChannelImpl.class
> >> >> WriteOnlyFileChannel.class
> >> >> LockManager.class
> >> >> LockManager$1.class
> >> >> ReadOnlyFileChannel.class
> >> >>
> >> >> case 2:
> >> >> ByteChannel.class
> >> >> Channel.class
> >> >> GatheringByteChannel.class
> >> >> InterruptibleChannel.class
> >> >> WritableByteChannel.class
> >> >>
> >> >> And I have got some ant script and more excluded list for emma, if
> >> >> anyone has interests, I can upload it to JIRA.
> >> >>
> >> >
> >> > Yes!
> >> >
> >> > -Mark.
> >> >
> >> >
> >> >> Mark Hindess wrote:
> >> >>
> >> >>> Anyone tried using emma (emma.sf.net) to look at test coverage
> >> for our
> >> >>> tests?  That might help ensure we don't trim them down too much?
> >> >>>
> >> >>> I suspect we will soon reach a point where there are significantly
> >> fewer
> >> >>> obvious opportunities for small contributions - e.g.
> >> 1.5generification,
> >> >>> etc from looking at JAPI reports.  Having test coverage reports
> >> might
> >> >>> help by highlighting areas for improvement.
> >> >>>
> >> >>> I had a quick try with:
> >> >>>
> >> >>>   ant -f make/build.xml
> >> >>>   ant -f modules/nio/make/build.xml test
> >> >>>   java -cp emma.jar emma instr -m overwrite -cp
> >> deploy/jdk/jre/lib/boot/nio
> >> >>>
> >> >> .jar
> >> >>
> >> >>> and then:
> >> >>>
> >> >>>   java -Xbootclasspath/p: emma.jar \
> >> >>>        -cp build/tests:modules/nio/bin/test:junit.jar:. \
> >> >>>        junit.textui.TestRunner \
> >> >>>        org.apache.harmony.tests.java.nio.channels.SocketChannelTest
> >> >>>
> >> >>> (Running all the tests is a little tricky because test depends
on
> >> build,
> >> >>> which depends on clean which wipes out the instr'd jar. ;-( )
> >> >>>
> >> >>> Sadly it dies horribly with the stack trace below.  I'll take
> >> another
> >> >>> look when I have a little more time, but I thought I'd mention
it
> in
> >> >>> case anyone else had any thoughts on the subject.
> >> >>>
> >> >>> Regards,
> >> >>>
> >> >>> Mark.
> >> >>>
> >> >>> Exception in thread "main" java/lang/ExceptionInInitializerError
> >> >>>         at java/lang/J9VMInternals.initialize
> >> (J9VMInternals.java:203)
> >> >>>         at java/nio/Buffer.$VRi ( Buffer.java:4294967295)
> >> >>>         at java/nio/Buffer.<clinit> ( Buffer.java:4294967295)
> >> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
> >> >>>         at java/lang/J9VMInternals.initialize ( J9VMInternals.java
> >> :185)
> >> >>>         at java/lang/J9VMInternals.initialize
> >> (J9VMInternals.java:154)
> >> >>>         at java/lang/String.getBytes (String.java:783)
> >> >>>         at org/apache/harmony/luni/util/Util.getBytes (
> >> Util.java:54)
> >> >>>         at java/lang/ClassLoader.loadLibraryWithPath
> >> (ClassLoader.java
> >> :998)
> >> >>>         at java/lang/ClassLoader.loadLibraryWithClassLoader (
> >> ClassLoader.ja
> >> >>>
> >> >> va:970)
> >> >>
> >> >>>         at java/lang/System.loadLibrary (System.java:706)
> >> >>>         at com/ibm/icu4jni/common/ICU4JNILoader.loadLibrary
> (Unknown
> >> Source
> >> >>>
> >> >> )
> >> >>
> >> >>>         at com/ibm/icu4jni/converters/NativeConverter.<clinit>
> >> (Unknown Sou
> >> >>>
> >> >> rce)
> >> >>
> >> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
> >> >>>         at java/lang/J9VMInternals.initialize
> >> ( J9VMInternals.java:185)
> >> >>>         at
> com/ibm/icu4jni/charset/CharsetProviderICU.charsetForName
> >> (Unkno
> >> >>>
> >> >> wn Source)
> >> >>
> >> >>>         at java/nio/charset/Charset.forNameInternal
> >> (Charset.java:450)
> >> >>>         at java/nio/charset/Charset.forName (Charset.java:513)
> >> >>>         at java/lang/String.defaultCharset ( String.java:688)
> >> >>>         at java/lang/String.<init> ( String.java:252)
> >> >>>         at org/apache/harmony/luni/util/Util.toString (Util.java
> :74)
> >> >>>         at java/lang/System.getPropertyList (Native Method)
> >> >>>         at java/lang/System.ensureProperties ( System.java:496)
> >> >>>         at java/lang/System.<clinit> (System.java:55)
> >> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
> >> >>>         at java/lang/J9VMInternals.initialize
> >> (J9VMInternals.java:185)
> >> >>>         at java/lang/ClassLoader.initializeClassLoaders (
> >> ClassLoader.java:6
> >> >>>
> >> >> 5)
> >> >>
> >> >>>         at java/lang/Thread.initialize (Thread.java:301)
> >> >>>         at java/lang/Thread.<init> (Thread.java:120)
> >> >>> java/lang/NullPointerException
> >> >>>         at java/lang/Throwable.printStackTrace ( Throwable.java
> :231)
> >> >>>         at com/vladium/emma/rt/RTExitHook.createClassLoaderClosure
> >> (RTExitH
> >> >>>
> >> >> ook.java:63)
> >> >>
> >> >>>         at com/vladium/emma/rt/RT.reset ( RT.java:92)
> >> >>>         at com/vladium/emma/rt/RT.<clinit> (RT.java:238)
> >> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
> >> >>>         at java/lang/J9VMInternals.initialize ( J9VMInternals.java
> >> :185)
> >> >>>         at java/nio/Buffer.$VRi ( Buffer.java:4294967295)
> >> >>>         at java/nio/Buffer.<clinit> (Buffer.java:4294967295)
> >> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
> >> >>>         at java/lang/J9VMInternals.initialize
> >> (J9VMInternals.java:185)
> >> >>>         at java/lang/J9VMInternals.initialize
> >> (J9VMInternals.java:154)
> >> >>>         at java/lang/String.getBytes ( String.java :783)
> >> >>>         at org/apache/harmony/luni/util/Util.getBytes (Util.java
> :54)
> >> >>>         at java/lang/ClassLoader.loadLibraryWithPath
> >> (ClassLoader.java
> >> :998)
> >> >>>         at java/lang/ClassLoader.loadLibraryWithClassLoader (
> >> ClassLoader.ja
> >> >>>
> >> >> va:970)
> >> >>
> >> >>>         at java/lang/System.loadLibrary ( System.java:706)
> >> >>>         at com/ibm/icu4jni/common/ICU4JNILoader.loadLibrary
> (Unknown
> >> Source
> >> >>>
> >> >> )
> >> >>
> >> >>>         at com/ibm/icu4jni/converters/NativeConverter.<clinit>
> >> (Unknown Sou
> >> >>>
> >> >> rce)
> >> >>
> >> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
> >> >>>         at java/lang/J9VMInternals.initialize
> >> (J9VMInternals.java:185)
> >> >>>         at
> com/ibm/icu4jni/charset/CharsetProviderICU.charsetForName
> >> (Unkno
> >> >>>
> >> >> wn Source)
> >> >>
> >> >>>         at java/nio/charset/Charset.forNameInternal
> >> (Charset.java:450)
> >> >>>         at java/nio/charset/Charset.forName (Charset.java:513)
> >> >>>         at java/lang/String.defaultCharset ( String.java:688)
> >> >>>         at java/lang/String.<init> (String.java:252)
> >> >>>         at org/apache/harmony/luni/util/Util.toString (Util.java
> :74)
> >> >>>         at java/lang/System.getPropertyList (Native Method)
> >> >>>         at java/lang/System.ensureProperties (System.java:496)
> >> >>>         at java/lang/System.<clinit> (System.java:55)
> >> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
> >> >>>         at java/lang/J9VMInternals.initialize
> >> (J9VMInternals.java:185)
> >> >>>         at java/lang/ClassLoader.initializeClassLoaders (
> >> ClassLoader.java:6
> >> >>>
> >> >> 5)
> >> >>
> >> >>>         at java/lang/Thread.initialize (Thread.java:301)
> >> >>>         at java/lang/Thread.<init> (Thread.java :120)
> >> >>> JVMJ9VM015W Initialization error for library jclclear_23(14):
> >> JVMJ9VM009E J
> >> >>>
> >> >> 9VMDllMain failed
> >> >>
> >> >>> HMYEXEL062E Internal VM error: Failed to create Java VM
> >> >>> FAILED.
> >> >>>
> >> >>>
> >> >>> On 1 June 2006 at 22:26, "Mikhail Loenko" < mloenko@gmail.com
>
> >> wrote:
> >> >>>
> >> >>>
> >> >>>> 2006/6/1, Geir Magnusson Jr < geir@pobox.com>:
> >> >>>>
> >> >>>>
> >> >>>>> Mikhail Loenko wrote:
> >> >>>>>
> >> >>>>>
> >> >>>>>> 2006/6/1, Geir Magnusson Jr < geir@pobox.com >:
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>> I don't quite grok what you mean - could you post
what you are
> >> >>>>>>> suggesting in code here on -dev?
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>> Index:
> >> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.ja
> >> >>>>>>
> >> >> v
> >> >>
> >> >>>>>>
> >> >>>>>>
> >> >>>> a
> >> >>>>
> >> >>>>
> >> >>>>>>
> >> ===================================================================
> >> >>>>>> ---
> >> >>>>>>
> >> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.java
> >> >>>>>> (revision
> >> >>>>>> 410480)
> >> >>>>>> +++
> >> >>>>>>
> >> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.java
> >>
> >> >>>>>> (working
> >> >>>>>> copy)
> >> >>>>>> @@ -785,7 +785,7 @@
> >> >>>>>>                 "FFFF:FFFF:FFFF:FFFF:FFFF:FFFF: 255.255.255.255
> ",
> >> >>>>>>                 "0:0:0:0:0:0:0:0", "0:0:0:0:0:0: 0.0.0.0"
};
> >> >>>>>>
> >> >>>>>> -        String invalidIPAddresses[] = { ":", "FFFF:FFFF",
> "1:1",
> >> >>>>>> "::1.2.3.444" };
> >> >>>>>> +        String invalidIPAddresses[] = { "FFFF:FFFF"
};
> >> >>>>>>
> >> >>>>>>         for (int i = 0; i < validIPAddresses.length;
i++) {
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>
> >> >>>>> Thx - why do you want to remove testing the others?  Because
you
> >> believe
> >> >>>>> that it is in effect testing the resolver in the OS since
each
> >> returns
> >> >>>>> the same answer,and thus it's testing the same thing over
and
> >> over?
> >> >>>>>
> >> >>>>>
> >> >>>> I believe so, but I'd like to hear confirmation from code authors
> >> >>>>
> >> >>>> Thanks,
> >> >>>> Mikhail
> >> >>>>
> >> >>>>
> >> >>>
> >> >>>
> >> ---------------------------------------------------------------------
> >> >>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >> >>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>
> >> >>> For additional commands, e-mail:
> >> harmony-dev-help@incubator.apache.org
> >> >>>
> >> >>>
> >> >>>
> >> >>>
> >> >> --
> >> >> Paulex Yang
> >> >> China Software Development Lab
> >> >> IBM
> >> >>
> >> >>
> >> >>
> >> >>
> ---------------------------------------------------------------------
> >> >> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >> >> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> >> >> For additional commands, e-mail:
> >> harmony-dev-help@incubator.apache.org
> >> >>
> >> >
> >> >
> >> >
> >> > ---------------------------------------------------------------------
>
> >> > Terms of use : http://incubator.apache.org/harmony/mailing.html
> >> > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> >> > For additional commands, e-mail:
> harmony-dev-help@incubator.apache.org
> >> >
> >> >
> >> >
> >>
> >>
> >> --
> >> Paulex Yang
> >> China Software Development Lab
> >> IBM
> >>
> >>
> >>
> >> ---------------------------------------------------------------------
> >> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> >> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >>
> >>
> >
>
>
> --
> Paulex Yang
> China Software Development Lab
> IBM
>
>
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message