harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paulex Yang <paulex.y...@gmail.com>
Subject Re: [classlib] [testing] Coverage (was Re: 37% of total test execution time is spent in a single test)
Date Tue, 13 Jun 2006 11:30:31 GMT
Vladimir Ivanov wrote:
> Latest Harmony API source coverage by Harmony API unit tests results I
> stored at wiki page
>
> http://wiki.apache.org/harmony/Coverage_information
Cool! Thank you, Vladimir.

Would you please also add the excluded class list, so that we can 
differentiate them from the classes actually without unit tests?
>
>
>
> I'm going to refresh it bi-weekly (seems, it is enough for coverage).
>
>
>
>> I think we have got a agreement on the test naming convention[1], but
>> for sure, there have been many (legacy) test cases before this
>> agreement, and I think the volunteer is highly welcome to provide patch
>> for them.
>>
>> [1]http://incubator.apache.org/harmony/subcomponents/classlibrary/testing.html 
>>
>
>>
>
> If nobody objects I'm going to look through the unit tests to correct
> package names according to the agreement (where needed).
>
>
>
> Thanks,
>
>   Vladimir
>
>
>
> On 6/8/06, Paulex Yang <paulex.yang@gmail.com > wrote:
>>
>> Vladimir,
>>
>> Vladimir Ivanov wrote:
>> > Thanks Paulex!
>> >
>> > I did the same, but could not send results due to spam filter J
>> > Observations:
>> >
>> >   1. Coverage results look pretty much similar.
>> >   2. Exclude list looks pretty much similar too, but, looks like it
>> >   depends on the way of data collection (I didn't run ant task and the
>> > list is
>> >   a little bit different).
>> Great.
>> >
>> > In any case, I think, when we run harmony on another VM exclude list
>> will
>> > have to be updated.
>> >
>> >
>> >
>> > May be we can start publishing the coverage information on wiki pages
>> and
>> > provide some updates time to time (I can do it)?
>> >
>> +1,  and of course, you can only if  no one in the mailing list objects
>> and you'll have my welcome, and I think it will be even greater if these
>> reports can be generated regularly like what JAPI is doing:)
>> >
>> >
>> > One note:
>> >
>> > I noticed that different unit tests have very different package names
>> >
>> > Now the directory with all built tests copied to one place looks like:
>> I think we have got a agreement on the test naming convention[1], but
>> for sure, there have been many (legacy) test cases before this
>> agreement, and I think the volunteer is highly welcome to provide patch
>> for them.
>>
>> [1]http://incubator.apache.org/harmony/subcomponents/classlibrary/testing.html 
>>
>>
>> >
>> >
>> >
>> > C:\coverage\tests\test>ls
>> >
>> > GZIPOutClose2.txt        api           config         javax
>> > tests
>> >
>> > GZIPOutFinish.txt        api.injected  dazzle         
>> org            xml
>>
>> >
>> > GZIPOutWrite.txt         binary        gif            prefs
>> >
>> > Inet6Address.golden.ser  bundles       impl           serialization
>> >
>> > JDK2-3gabba.zip          com           impl.injected   test.txt
>> >
>> >
>> >
>> > I think, it would be good if tests had unified package names.
>> >
>> > Why? – so far, just common sense, just to have an order in test suite
>> >
>> > Organization (if consider all unit tests as solid test suite).
>> >
>> > Thanks,
>> >  Vladimir
>> >
>> > For example, my exclude list for java.io is:
>> > -java.io.BufferedInputStream ,
>> > -java.io.BufferedOutputStream,
>> > -java.io.File ,
>> > -java.io.FileChannelFactory,
>> > -java.io.FileDescriptor,
>> > -java.io.FileInputStream,
>> > -java.io.FileOutputStream,
>> > -java.io.FilterInputStream ,
>> > -java.io.FilterOutputStream,
>> > - java.io.InputStream,
>> > -java.io.OutputStream,
>> > -java.io.ObjectStreamField,
>> > -java.io.PrintStream
>> >
>> >
>> > On 6/6/06, Paulex Yang < paulex.yang@gmail.com > wrote:
>> >>
>> >> I've attach the scripts and excluded class lists to JIRA,  please 
>> refer
>> >> to https://issues.apache.org/jira/browse/HARMONY-564 .  Enjoy it:).
>> >>
>> >> Mark Hindess wrote:
>> >> > On 2 June 2006 at 10:37, Paulex Yang < paulex.yang@gmail.com>

>> wrote:
>> >> >
>> >> >> Mark,
>> >> >>
>> >> >> I'm glad that there is someone else has interest on emma, I've
>> >> tried it
>> >> >> before. AFAIK, emma works by instrumentation, but sometimes for
>> >> classes
>> >> >> in bootclasspath, the instrumentation cannot work, there are two
>> >> cases:
>> >>
>> >> >> 1. Some instrumented classes cannot be loaded by VM.
>> >> >> 2. Some classes cannot be instrumented
>> >> >>
>> >> >> I have tried to look more inside to find some way to work around
>> >> but I
>> >> >> haven't got enough time yet.
>> >> >>
>> >> >> Specifically for nio-channel module, I had a list for these two
>> cases
>> >> (I
>> >> >> believe the data is a little outdated and should be reevaluated)
>> >> >> case 1.
>> >> >> BaseByteBuffer.class
>> >> >> Buffer.class
>> >> >> BufferFactory.class
>> >> >> ByteBuffer.class
>> >> >> CharArrayBuffer.class
>> >> >> CharBuffer.class
>> >> >> HeapByteBuffer.class
>> >> >> ReadWriteCharArrayBuffer.class
>> >> >> ReadWriteHeapByteBuffer.class
>> >> >> FileChannel.class
>> >> >> AbstractInterruptibleChannel.class
>> >> >> FileChannelImpl.class
>> >> >> WriteOnlyFileChannel.class
>> >> >> LockManager.class
>> >> >> LockManager$1.class
>> >> >> ReadOnlyFileChannel.class
>> >> >>
>> >> >> case 2:
>> >> >> ByteChannel.class
>> >> >> Channel.class
>> >> >> GatheringByteChannel.class
>> >> >> InterruptibleChannel.class
>> >> >> WritableByteChannel.class
>> >> >>
>> >> >> And I have got some ant script and more excluded list for emma,
if
>> >> >> anyone has interests, I can upload it to JIRA.
>> >> >>
>> >> >
>> >> > Yes!
>> >> >
>> >> > -Mark.
>> >> >
>> >> >
>> >> >> Mark Hindess wrote:
>> >> >>
>> >> >>> Anyone tried using emma (emma.sf.net) to look at test coverage
>> >> for our
>> >> >>> tests?  That might help ensure we don't trim them down too
much?
>> >> >>>
>> >> >>> I suspect we will soon reach a point where there are 
>> significantly
>> >> fewer
>> >> >>> obvious opportunities for small contributions - e.g.
>> >> 1.5generification,
>> >> >>> etc from looking at JAPI reports.  Having test coverage reports
>> >> might
>> >> >>> help by highlighting areas for improvement.
>> >> >>>
>> >> >>> I had a quick try with:
>> >> >>>
>> >> >>>   ant -f make/build.xml
>> >> >>>   ant -f modules/nio/make/build.xml test
>> >> >>>   java -cp emma.jar emma instr -m overwrite -cp
>> >> deploy/jdk/jre/lib/boot/nio
>> >> >>>
>> >> >> .jar
>> >> >>
>> >> >>> and then:
>> >> >>>
>> >> >>>   java -Xbootclasspath/p: emma.jar \
>> >> >>>        -cp build/tests:modules/nio/bin/test:junit.jar:. \
>> >> >>>        junit.textui.TestRunner \
>> >> >>>        
>> org.apache.harmony.tests.java.nio.channels.SocketChannelTest
>> >> >>>
>> >> >>> (Running all the tests is a little tricky because test depends
on
>> >> build,
>> >> >>> which depends on clean which wipes out the instr'd jar. ;-(
)
>> >> >>>
>> >> >>> Sadly it dies horribly with the stack trace below.  I'll take
>> >> another
>> >> >>> look when I have a little more time, but I thought I'd mention
it
>> in
>> >> >>> case anyone else had any thoughts on the subject.
>> >> >>>
>> >> >>> Regards,
>> >> >>>
>> >> >>> Mark.
>> >> >>>
>> >> >>> Exception in thread "main" java/lang/ExceptionInInitializerError
>> >> >>>         at java/lang/J9VMInternals.initialize
>> >> (J9VMInternals.java:203)
>> >> >>>         at java/nio/Buffer.$VRi ( Buffer.java:4294967295)
>> >> >>>         at java/nio/Buffer.<clinit> ( Buffer.java:4294967295)
>> >> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
>> >> >>>         at java/lang/J9VMInternals.initialize ( 
>> J9VMInternals.java
>> >> :185)
>> >> >>>         at java/lang/J9VMInternals.initialize
>> >> (J9VMInternals.java:154)
>> >> >>>         at java/lang/String.getBytes (String.java:783)
>> >> >>>         at org/apache/harmony/luni/util/Util.getBytes (
>> >> Util.java:54)
>> >> >>>         at java/lang/ClassLoader.loadLibraryWithPath
>> >> (ClassLoader.java
>> >> :998)
>> >> >>>         at java/lang/ClassLoader.loadLibraryWithClassLoader
(
>> >> ClassLoader.ja
>> >> >>>
>> >> >> va:970)
>> >> >>
>> >> >>>         at java/lang/System.loadLibrary (System.java:706)
>> >> >>>         at com/ibm/icu4jni/common/ICU4JNILoader.loadLibrary
>> (Unknown
>> >> Source
>> >> >>>
>> >> >> )
>> >> >>
>> >> >>>         at com/ibm/icu4jni/converters/NativeConverter.<clinit>
>> >> (Unknown Sou
>> >> >>>
>> >> >> rce)
>> >> >>
>> >> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
>> >> >>>         at java/lang/J9VMInternals.initialize
>> >> ( J9VMInternals.java:185)
>> >> >>>         at
>> com/ibm/icu4jni/charset/CharsetProviderICU.charsetForName
>> >> (Unkno
>> >> >>>
>> >> >> wn Source)
>> >> >>
>> >> >>>         at java/nio/charset/Charset.forNameInternal
>> >> (Charset.java:450)
>> >> >>>         at java/nio/charset/Charset.forName (Charset.java:513)
>> >> >>>         at java/lang/String.defaultCharset ( String.java:688)
>> >> >>>         at java/lang/String.<init> ( String.java:252)
>> >> >>>         at org/apache/harmony/luni/util/Util.toString (Util.java
>> :74)
>> >> >>>         at java/lang/System.getPropertyList (Native Method)
>> >> >>>         at java/lang/System.ensureProperties ( System.java:496)
>> >> >>>         at java/lang/System.<clinit> (System.java:55)
>> >> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
>> >> >>>         at java/lang/J9VMInternals.initialize
>> >> (J9VMInternals.java:185)
>> >> >>>         at java/lang/ClassLoader.initializeClassLoaders (
>> >> ClassLoader.java:6
>> >> >>>
>> >> >> 5)
>> >> >>
>> >> >>>         at java/lang/Thread.initialize (Thread.java:301)
>> >> >>>         at java/lang/Thread.<init> (Thread.java:120)
>> >> >>> java/lang/NullPointerException
>> >> >>>         at java/lang/Throwable.printStackTrace ( Throwable.java
>> :231)
>> >> >>>         at 
>> com/vladium/emma/rt/RTExitHook.createClassLoaderClosure
>> >> (RTExitH
>> >> >>>
>> >> >> ook.java:63)
>> >> >>
>> >> >>>         at com/vladium/emma/rt/RT.reset ( RT.java:92)
>> >> >>>         at com/vladium/emma/rt/RT.<clinit> (RT.java:238)
>> >> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
>> >> >>>         at java/lang/J9VMInternals.initialize ( 
>> J9VMInternals.java
>> >> :185)
>> >> >>>         at java/nio/Buffer.$VRi ( Buffer.java:4294967295)
>> >> >>>         at java/nio/Buffer.<clinit> (Buffer.java:4294967295)
>> >> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
>> >> >>>         at java/lang/J9VMInternals.initialize
>> >> (J9VMInternals.java:185)
>> >> >>>         at java/lang/J9VMInternals.initialize
>> >> (J9VMInternals.java:154)
>> >> >>>         at java/lang/String.getBytes ( String.java :783)
>> >> >>>         at org/apache/harmony/luni/util/Util.getBytes (Util.java
>> :54)
>> >> >>>         at java/lang/ClassLoader.loadLibraryWithPath
>> >> (ClassLoader.java
>> >> :998)
>> >> >>>         at java/lang/ClassLoader.loadLibraryWithClassLoader
(
>> >> ClassLoader.ja
>> >> >>>
>> >> >> va:970)
>> >> >>
>> >> >>>         at java/lang/System.loadLibrary ( System.java:706)
>> >> >>>         at com/ibm/icu4jni/common/ICU4JNILoader.loadLibrary
>> (Unknown
>> >> Source
>> >> >>>
>> >> >> )
>> >> >>
>> >> >>>         at com/ibm/icu4jni/converters/NativeConverter.<clinit>
>> >> (Unknown Sou
>> >> >>>
>> >> >> rce)
>> >> >>
>> >> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
>> >> >>>         at java/lang/J9VMInternals.initialize
>> >> (J9VMInternals.java:185)
>> >> >>>         at
>> com/ibm/icu4jni/charset/CharsetProviderICU.charsetForName
>> >> (Unkno
>> >> >>>
>> >> >> wn Source)
>> >> >>
>> >> >>>         at java/nio/charset/Charset.forNameInternal
>> >> (Charset.java:450)
>> >> >>>         at java/nio/charset/Charset.forName (Charset.java:513)
>> >> >>>         at java/lang/String.defaultCharset ( String.java:688)
>> >> >>>         at java/lang/String.<init> (String.java:252)
>> >> >>>         at org/apache/harmony/luni/util/Util.toString (Util.java
>> :74)
>> >> >>>         at java/lang/System.getPropertyList (Native Method)
>> >> >>>         at java/lang/System.ensureProperties (System.java:496)
>> >> >>>         at java/lang/System.<clinit> (System.java:55)
>> >> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
>> >> >>>         at java/lang/J9VMInternals.initialize
>> >> (J9VMInternals.java:185)
>> >> >>>         at java/lang/ClassLoader.initializeClassLoaders (
>> >> ClassLoader.java:6
>> >> >>>
>> >> >> 5)
>> >> >>
>> >> >>>         at java/lang/Thread.initialize (Thread.java:301)
>> >> >>>         at java/lang/Thread.<init> (Thread.java :120)
>> >> >>> JVMJ9VM015W Initialization error for library jclclear_23(14):
>> >> JVMJ9VM009E J
>> >> >>>
>> >> >> 9VMDllMain failed
>> >> >>
>> >> >>> HMYEXEL062E Internal VM error: Failed to create Java VM
>> >> >>> FAILED.
>> >> >>>
>> >> >>>
>> >> >>> On 1 June 2006 at 22:26, "Mikhail Loenko" < mloenko@gmail.com
>
>> >> wrote:
>> >> >>>
>> >> >>>
>> >> >>>> 2006/6/1, Geir Magnusson Jr < geir@pobox.com>:
>> >> >>>>
>> >> >>>>
>> >> >>>>> Mikhail Loenko wrote:
>> >> >>>>>
>> >> >>>>>
>> >> >>>>>> 2006/6/1, Geir Magnusson Jr < geir@pobox.com
>:
>> >> >>>>>>
>> >> >>>>>>
>> >> >>>>>>> I don't quite grok what you mean - could you
post what you 
>> are
>> >> >>>>>>> suggesting in code here on -dev?
>> >> >>>>>>>
>> >> >>>>>>>
>> >> >>>>>> Index:
>> >> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.ja
>> >> >>>>>>
>> >> >> v
>> >> >>
>> >> >>>>>>
>> >> >>>>>>
>> >> >>>> a
>> >> >>>>
>> >> >>>>
>> >> >>>>>>
>> >> ===================================================================
>> >> >>>>>> ---
>> >> >>>>>>
>> >> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.java
>> >> >>>>>> (revision
>> >> >>>>>> 410480)
>> >> >>>>>> +++
>> >> >>>>>>
>> >> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.java
>> >>
>> >> >>>>>> (working
>> >> >>>>>> copy)
>> >> >>>>>> @@ -785,7 +785,7 @@
>> >> >>>>>>                 "FFFF:FFFF:FFFF:FFFF:FFFF:FFFF:

>> 255.255.255.255
>> ",
>> >> >>>>>>                 "0:0:0:0:0:0:0:0", "0:0:0:0:0:0:
0.0.0.0" };
>> >> >>>>>>
>> >> >>>>>> -        String invalidIPAddresses[] = { ":", "FFFF:FFFF",
>> "1:1",
>> >> >>>>>> "::1.2.3.444" };
>> >> >>>>>> +        String invalidIPAddresses[] = { "FFFF:FFFF"
};
>> >> >>>>>>
>> >> >>>>>>         for (int i = 0; i < validIPAddresses.length;
i++) {
>> >> >>>>>>
>> >> >>>>>>
>> >> >>>>>>
>> >> >>>>>>
>> >> >>>>> Thx - why do you want to remove testing the others?
 Because 
>> you
>> >> believe
>> >> >>>>> that it is in effect testing the resolver in the OS
since each
>> >> returns
>> >> >>>>> the same answer,and thus it's testing the same thing
over and
>> >> over?
>> >> >>>>>
>> >> >>>>>
>> >> >>>> I believe so, but I'd like to hear confirmation from code

>> authors
>> >> >>>>
>> >> >>>> Thanks,
>> >> >>>> Mikhail
>> >> >>>>
>> >> >>>>
>> >> >>>
>> >> >>>
>> >> ---------------------------------------------------------------------
>> >> >>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>> >> >>> To unsubscribe, e-mail: 
>> harmony-dev-unsubscribe@incubator.apache.org
>>
>> >> >>> For additional commands, e-mail:
>> >> harmony-dev-help@incubator.apache.org
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >> --
>> >> >> Paulex Yang
>> >> >> China Software Development Lab
>> >> >> IBM
>> >> >>
>> >> >>
>> >> >>
>> >> >>
>> ---------------------------------------------------------------------
>> >> >> Terms of use : http://incubator.apache.org/harmony/mailing.html
>> >> >> To unsubscribe, e-mail: 
>> harmony-dev-unsubscribe@incubator.apache.org
>> >> >> For additional commands, e-mail:
>> >> harmony-dev-help@incubator.apache.org
>> >> >>
>> >> >
>> >> >
>> >> >
>> >> > 
>> ---------------------------------------------------------------------
>>
>> >> > Terms of use : http://incubator.apache.org/harmony/mailing.html
>> >> > To unsubscribe, e-mail: 
>> harmony-dev-unsubscribe@incubator.apache.org
>> >> > For additional commands, e-mail:
>> harmony-dev-help@incubator.apache.org
>> >> >
>> >> >
>> >> >
>> >>
>> >>
>> >> --
>> >> Paulex Yang
>> >> China Software Development Lab
>> >> IBM
>> >>
>> >>
>> >>
>> >> ---------------------------------------------------------------------
>> >> Terms of use : http://incubator.apache.org/harmony/mailing.html
>> >> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>> >> For additional commands, e-mail: 
>> harmony-dev-help@incubator.apache.org
>> >>
>> >>
>> >
>>
>>
>> -- 
>> Paulex Yang
>> China Software Development Lab
>> IBM
>>
>>
>>
>> ---------------------------------------------------------------------
>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>
>>
>


-- 
Paulex Yang
China Software Development Lab
IBM



---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org


Mime
View raw message