harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Geir Magnusson Jr <g...@pobox.com>
Subject Re: [classlib] [testing] Coverage (was Re: 37% of total test execution time is spent in a single test)
Date Wed, 14 Jun 2006 11:04:24 GMT
yes.

And/or add it to the upcoming donation of the [testing] infrastructure
to make it part of the CI runs.

geir

Nathan Beyer wrote:
> Is there any possibility of adding this to an Ant script that can optionally
> be run with the build scripts? I'd like to make this easier for everyone to
> run while developing.
> 
> Has anyone tried any other coverage tools? Like TPTP for Eclipse, Clover or
> Corbetura (sp)?
> 
> -Nathan
> 
>> -----Original Message-----
>> From: Vladimir Ivanov [mailto:ivavladimir@gmail.com]
>>
>> Latest Harmony API source coverage by Harmony API unit tests results I
>> stored at wiki page
>>
>> http://wiki.apache.org/harmony/Coverage_information
>>
>>
>>
>> I'm going to refresh it bi-weekly (seems, it is enough for coverage).
>>
>>
>>
>>> I think we have got a agreement on the test naming convention[1], but
>>> for sure, there have been many (legacy) test cases before this
>>> agreement, and I think the volunteer is highly welcome to provide patch
>>> for them.
>>>
>>> [1]http://incubator.apache.org/harmony/subcomponents/classlibrary/testing
>> .html
>>
>> If nobody objects I'm going to look through the unit tests to correct
>> package names according to the agreement (where needed).
>>
>>
>>
>>  Thanks,
>>
>>    Vladimir
>>
>>
>>
>> On 6/8/06, Paulex Yang <paulex.yang@gmail.com > wrote:
>>> Vladimir,
>>>
>>> Vladimir Ivanov wrote:
>>>> Thanks Paulex!
>>>>
>>>> I did the same, but could not send results due to spam filter J
>>>> Observations:
>>>>
>>>>   1. Coverage results look pretty much similar.
>>>>   2. Exclude list looks pretty much similar too, but, looks like it
>>>>   depends on the way of data collection (I didn't run ant task and the
>>>> list is
>>>>   a little bit different).
>>> Great.
>>>> In any case, I think, when we run harmony on another VM exclude list
>>> will
>>>> have to be updated.
>>>>
>>>>
>>>>
>>>> May be we can start publishing the coverage information on wiki pages
>>> and
>>>> provide some updates time to time (I can do it)?
>>>>
>>> +1,  and of course, you can only if  no one in the mailing list objects
>>> and you'll have my welcome, and I think it will be even greater if these
>>> reports can be generated regularly like what JAPI is doing:)
>>>>
>>>> One note:
>>>>
>>>> I noticed that different unit tests have very different package names
>>>>
>>>> Now the directory with all built tests copied to one place looks like:
>>> I think we have got a agreement on the test naming convention[1], but
>>> for sure, there have been many (legacy) test cases before this
>>> agreement, and I think the volunteer is highly welcome to provide patch
>>> for them.
>>>
>>>
>> [1]http://incubator.apache.org/harmony/subcomponents/classlibrary/testing.
>> html
>>>>
>>>>
>>>> C:\coverage\tests\test>ls
>>>>
>>>> GZIPOutClose2.txt        api           config         javax
>>>> tests
>>>>
>>>> GZIPOutFinish.txt        api.injected  dazzle         org
>> xml
>>>> GZIPOutWrite.txt         binary        gif            prefs
>>>>
>>>> Inet6Address.golden.ser  bundles       impl           serialization
>>>>
>>>> JDK2-3gabba.zip          com           impl.injected   test.txt
>>>>
>>>>
>>>>
>>>> I think, it would be good if tests had unified package names.
>>>>
>>>> Why? - so far, just common sense, just to have an order in test suite
>>>>
>>>> Organization (if consider all unit tests as solid test suite).
>>>>
>>>> Thanks,
>>>>  Vladimir
>>>>
>>>> For example, my exclude list for java.io is:
>>>> -java.io.BufferedInputStream ,
>>>> -java.io.BufferedOutputStream,
>>>> -java.io.File ,
>>>> -java.io.FileChannelFactory,
>>>> -java.io.FileDescriptor,
>>>> -java.io.FileInputStream,
>>>> -java.io.FileOutputStream,
>>>> -java.io.FilterInputStream ,
>>>> -java.io.FilterOutputStream,
>>>> - java.io.InputStream,
>>>> -java.io.OutputStream,
>>>> -java.io.ObjectStreamField,
>>>> -java.io.PrintStream
>>>>
>>>>
>>>> On 6/6/06, Paulex Yang < paulex.yang@gmail.com > wrote:
>>>>> I've attach the scripts and excluded class lists to JIRA,  please
>> refer
>>>>> to https://issues.apache.org/jira/browse/HARMONY-564 .  Enjoy it:).
>>>>>
>>>>> Mark Hindess wrote:
>>>>>> On 2 June 2006 at 10:37, Paulex Yang < paulex.yang@gmail.com>
>> wrote:
>>>>>>> Mark,
>>>>>>>
>>>>>>> I'm glad that there is someone else has interest on emma, I've
>>>>> tried it
>>>>>>> before. AFAIK, emma works by instrumentation, but sometimes for
>>>>> classes
>>>>>>> in bootclasspath, the instrumentation cannot work, there are
two
>>>>> cases:
>>>>>
>>>>>>> 1. Some instrumented classes cannot be loaded by VM.
>>>>>>> 2. Some classes cannot be instrumented
>>>>>>>
>>>>>>> I have tried to look more inside to find some way to work around
>>>>> but I
>>>>>>> haven't got enough time yet.
>>>>>>>
>>>>>>> Specifically for nio-channel module, I had a list for these two
>>> cases
>>>>> (I
>>>>>>> believe the data is a little outdated and should be reevaluated)
>>>>>>> case 1.
>>>>>>> BaseByteBuffer.class
>>>>>>> Buffer.class
>>>>>>> BufferFactory.class
>>>>>>> ByteBuffer.class
>>>>>>> CharArrayBuffer.class
>>>>>>> CharBuffer.class
>>>>>>> HeapByteBuffer.class
>>>>>>> ReadWriteCharArrayBuffer.class
>>>>>>> ReadWriteHeapByteBuffer.class
>>>>>>> FileChannel.class
>>>>>>> AbstractInterruptibleChannel.class
>>>>>>> FileChannelImpl.class
>>>>>>> WriteOnlyFileChannel.class
>>>>>>> LockManager.class
>>>>>>> LockManager$1.class
>>>>>>> ReadOnlyFileChannel.class
>>>>>>>
>>>>>>> case 2:
>>>>>>> ByteChannel.class
>>>>>>> Channel.class
>>>>>>> GatheringByteChannel.class
>>>>>>> InterruptibleChannel.class
>>>>>>> WritableByteChannel.class
>>>>>>>
>>>>>>> And I have got some ant script and more excluded list for emma,
if
>>>>>>> anyone has interests, I can upload it to JIRA.
>>>>>>>
>>>>>> Yes!
>>>>>>
>>>>>> -Mark.
>>>>>>
>>>>>>
>>>>>>> Mark Hindess wrote:
>>>>>>>
>>>>>>>> Anyone tried using emma (emma.sf.net) to look at test coverage
>>>>> for our
>>>>>>>> tests?  That might help ensure we don't trim them down too
much?
>>>>>>>>
>>>>>>>> I suspect we will soon reach a point where there are
>> significantly
>>>>> fewer
>>>>>>>> obvious opportunities for small contributions - e.g.
>>>>> 1.5generification,
>>>>>>>> etc from looking at JAPI reports.  Having test coverage reports
>>>>> might
>>>>>>>> help by highlighting areas for improvement.
>>>>>>>>
>>>>>>>> I had a quick try with:
>>>>>>>>
>>>>>>>>   ant -f make/build.xml
>>>>>>>>   ant -f modules/nio/make/build.xml test
>>>>>>>>   java -cp emma.jar emma instr -m overwrite -cp
>>>>> deploy/jdk/jre/lib/boot/nio
>>>>>>> .jar
>>>>>>>
>>>>>>>> and then:
>>>>>>>>
>>>>>>>>   java -Xbootclasspath/p: emma.jar \
>>>>>>>>        -cp build/tests:modules/nio/bin/test:junit.jar:. \
>>>>>>>>        junit.textui.TestRunner \
>>>>>>>>
>> org.apache.harmony.tests.java.nio.channels.SocketChannelTest
>>>>>>>> (Running all the tests is a little tricky because test depends
on
>>>>> build,
>>>>>>>> which depends on clean which wipes out the instr'd jar. ;-(
)
>>>>>>>>
>>>>>>>> Sadly it dies horribly with the stack trace below.  I'll
take
>>>>> another
>>>>>>>> look when I have a little more time, but I thought I'd mention
it
>>> in
>>>>>>>> case anyone else had any thoughts on the subject.
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>>
>>>>>>>> Mark.
>>>>>>>>
>>>>>>>> Exception in thread "main" java/lang/ExceptionInInitializerError
>>>>>>>>         at java/lang/J9VMInternals.initialize
>>>>> (J9VMInternals.java:203)
>>>>>>>>         at java/nio/Buffer.$VRi ( Buffer.java:4294967295)
>>>>>>>>         at java/nio/Buffer.<clinit> ( Buffer.java:4294967295)
>>>>>>>>         at java/lang/J9VMInternals.initializeImpl (Native
Method)
>>>>>>>>         at java/lang/J9VMInternals.initialize (
>> J9VMInternals.java
>>>>> :185)
>>>>>>>>         at java/lang/J9VMInternals.initialize
>>>>> (J9VMInternals.java:154)
>>>>>>>>         at java/lang/String.getBytes (String.java:783)
>>>>>>>>         at org/apache/harmony/luni/util/Util.getBytes (
>>>>> Util.java:54)
>>>>>>>>         at java/lang/ClassLoader.loadLibraryWithPath
>>>>> (ClassLoader.java
>>>>> :998)
>>>>>>>>         at java/lang/ClassLoader.loadLibraryWithClassLoader
(
>>>>> ClassLoader.ja
>>>>>>> va:970)
>>>>>>>
>>>>>>>>         at java/lang/System.loadLibrary (System.java:706)
>>>>>>>>         at com/ibm/icu4jni/common/ICU4JNILoader.loadLibrary
>>> (Unknown
>>>>> Source
>>>>>>> )
>>>>>>>
>>>>>>>>         at com/ibm/icu4jni/converters/NativeConverter.<clinit>
>>>>> (Unknown Sou
>>>>>>> rce)
>>>>>>>
>>>>>>>>         at java/lang/J9VMInternals.initializeImpl (Native
Method)
>>>>>>>>         at java/lang/J9VMInternals.initialize
>>>>> ( J9VMInternals.java:185)
>>>>>>>>         at
>>> com/ibm/icu4jni/charset/CharsetProviderICU.charsetForName
>>>>> (Unkno
>>>>>>> wn Source)
>>>>>>>
>>>>>>>>         at java/nio/charset/Charset.forNameInternal
>>>>> (Charset.java:450)
>>>>>>>>         at java/nio/charset/Charset.forName (Charset.java:513)
>>>>>>>>         at java/lang/String.defaultCharset ( String.java:688)
>>>>>>>>         at java/lang/String.<init> ( String.java:252)
>>>>>>>>         at org/apache/harmony/luni/util/Util.toString (Util.java
>>> :74)
>>>>>>>>         at java/lang/System.getPropertyList (Native Method)
>>>>>>>>         at java/lang/System.ensureProperties ( System.java:496)
>>>>>>>>         at java/lang/System.<clinit> (System.java:55)
>>>>>>>>         at java/lang/J9VMInternals.initializeImpl (Native
Method)
>>>>>>>>         at java/lang/J9VMInternals.initialize
>>>>> (J9VMInternals.java:185)
>>>>>>>>         at java/lang/ClassLoader.initializeClassLoaders (
>>>>> ClassLoader.java:6
>>>>>>> 5)
>>>>>>>
>>>>>>>>         at java/lang/Thread.initialize (Thread.java:301)
>>>>>>>>         at java/lang/Thread.<init> (Thread.java:120)
>>>>>>>> java/lang/NullPointerException
>>>>>>>>         at java/lang/Throwable.printStackTrace ( Throwable.java
>>> :231)
>>>>>>>>         at
>> com/vladium/emma/rt/RTExitHook.createClassLoaderClosure
>>>>> (RTExitH
>>>>>>> ook.java:63)
>>>>>>>
>>>>>>>>         at com/vladium/emma/rt/RT.reset ( RT.java:92)
>>>>>>>>         at com/vladium/emma/rt/RT.<clinit> (RT.java:238)
>>>>>>>>         at java/lang/J9VMInternals.initializeImpl (Native
Method)
>>>>>>>>         at java/lang/J9VMInternals.initialize (
>> J9VMInternals.java
>>>>> :185)
>>>>>>>>         at java/nio/Buffer.$VRi ( Buffer.java:4294967295)
>>>>>>>>         at java/nio/Buffer.<clinit> (Buffer.java:4294967295)
>>>>>>>>         at java/lang/J9VMInternals.initializeImpl (Native
Method)
>>>>>>>>         at java/lang/J9VMInternals.initialize
>>>>> (J9VMInternals.java:185)
>>>>>>>>         at java/lang/J9VMInternals.initialize
>>>>> (J9VMInternals.java:154)
>>>>>>>>         at java/lang/String.getBytes ( String.java :783)
>>>>>>>>         at org/apache/harmony/luni/util/Util.getBytes (Util.java
>>> :54)
>>>>>>>>         at java/lang/ClassLoader.loadLibraryWithPath
>>>>> (ClassLoader.java
>>>>> :998)
>>>>>>>>         at java/lang/ClassLoader.loadLibraryWithClassLoader
(
>>>>> ClassLoader.ja
>>>>>>> va:970)
>>>>>>>
>>>>>>>>         at java/lang/System.loadLibrary ( System.java:706)
>>>>>>>>         at com/ibm/icu4jni/common/ICU4JNILoader.loadLibrary
>>> (Unknown
>>>>> Source
>>>>>>> )
>>>>>>>
>>>>>>>>         at com/ibm/icu4jni/converters/NativeConverter.<clinit>
>>>>> (Unknown Sou
>>>>>>> rce)
>>>>>>>
>>>>>>>>         at java/lang/J9VMInternals.initializeImpl (Native
Method)
>>>>>>>>         at java/lang/J9VMInternals.initialize
>>>>> (J9VMInternals.java:185)
>>>>>>>>         at
>>> com/ibm/icu4jni/charset/CharsetProviderICU.charsetForName
>>>>> (Unkno
>>>>>>> wn Source)
>>>>>>>
>>>>>>>>         at java/nio/charset/Charset.forNameInternal
>>>>> (Charset.java:450)
>>>>>>>>         at java/nio/charset/Charset.forName (Charset.java:513)
>>>>>>>>         at java/lang/String.defaultCharset ( String.java:688)
>>>>>>>>         at java/lang/String.<init> (String.java:252)
>>>>>>>>         at org/apache/harmony/luni/util/Util.toString (Util.java
>>> :74)
>>>>>>>>         at java/lang/System.getPropertyList (Native Method)
>>>>>>>>         at java/lang/System.ensureProperties (System.java:496)
>>>>>>>>         at java/lang/System.<clinit> (System.java:55)
>>>>>>>>         at java/lang/J9VMInternals.initializeImpl (Native
Method)
>>>>>>>>         at java/lang/J9VMInternals.initialize
>>>>> (J9VMInternals.java:185)
>>>>>>>>         at java/lang/ClassLoader.initializeClassLoaders (
>>>>> ClassLoader.java:6
>>>>>>> 5)
>>>>>>>
>>>>>>>>         at java/lang/Thread.initialize (Thread.java:301)
>>>>>>>>         at java/lang/Thread.<init> (Thread.java :120)
>>>>>>>> JVMJ9VM015W Initialization error for library jclclear_23(14):
>>>>> JVMJ9VM009E J
>>>>>>> 9VMDllMain failed
>>>>>>>
>>>>>>>> HMYEXEL062E Internal VM error: Failed to create Java VM
>>>>>>>> FAILED.
>>>>>>>>
>>>>>>>>
>>>>>>>> On 1 June 2006 at 22:26, "Mikhail Loenko" < mloenko@gmail.com
>
>>>>> wrote:
>>>>>>>>
>>>>>>>>> 2006/6/1, Geir Magnusson Jr < geir@pobox.com>:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> Mikhail Loenko wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> 2006/6/1, Geir Magnusson Jr < geir@pobox.com
>:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> I don't quite grok what you mean - could
you post what you
>> are
>>>>>>>>>>>> suggesting in code here on -dev?
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> Index:
>>>>> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.ja
>>>>>>> v
>>>>>>>
>>>>>>>>>>>
>>>>>>>>> a
>>>>>>>>>
>>>>>>>>>
>>>>> ===================================================================
>>>>>>>>>>> ---
>>>>>>>>>>>
>>>>> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.java
>>>>>>>>>>> (revision
>>>>>>>>>>> 410480)
>>>>>>>>>>> +++
>>>>>>>>>>>
>>>>> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.java
>>>>>
>>>>>>>>>>> (working
>>>>>>>>>>> copy)
>>>>>>>>>>> @@ -785,7 +785,7 @@
>>>>>>>>>>>                 "FFFF:FFFF:FFFF:FFFF:FFFF:FFFF:
>> 255.255.255.255
>>> ",
>>>>>>>>>>>                 "0:0:0:0:0:0:0:0", "0:0:0:0:0:0:
0.0.0.0" };
>>>>>>>>>>>
>>>>>>>>>>> -        String invalidIPAddresses[] = { ":",
"FFFF:FFFF",
>>> "1:1",
>>>>>>>>>>> "::1.2.3.444" };
>>>>>>>>>>> +        String invalidIPAddresses[] = { "FFFF:FFFF"
};
>>>>>>>>>>>
>>>>>>>>>>>         for (int i = 0; i < validIPAddresses.length;
i++) {
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>> Thx - why do you want to remove testing the others?
 Because
>> you
>>>>> believe
>>>>>>>>>> that it is in effect testing the resolver in the
OS since each
>>>>> returns
>>>>>>>>>> the same answer,and thus it's testing the same thing
over and
>>>>> over?
>>>>>>>>>>
>>>>>>>>> I believe so, but I'd like to hear confirmation from
code
>> authors
>>>>>>>>> Thanks,
>>>>>>>>> Mikhail
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>> ---------------------------------------------------------------------
>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>>> To unsubscribe, e-mail: harmony-dev-
>> unsubscribe@incubator.apache.org
>>>>>>>> For additional commands, e-mail:
>>>>> harmony-dev-help@incubator.apache.org
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>> --
>>>>>>> Paulex Yang
>>>>>>> China Software Development Lab
>>>>>>> IBM
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>> ---------------------------------------------------------------------
>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>> To unsubscribe, e-mail: harmony-dev-
>> unsubscribe@incubator.apache.org
>>>>>>> For additional commands, e-mail:
>>>>> harmony-dev-help@incubator.apache.org
>>>>>>
>>>>>>
>>>>>> -------------------------------------------------------------------
>> --
>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>> To unsubscribe, e-mail: harmony-dev-
>> unsubscribe@incubator.apache.org
>>>>>> For additional commands, e-mail:
>>> harmony-dev-help@incubator.apache.org
>>>>>>
>>>>>>
>>>>>
>>>>> --
>>>>> Paulex Yang
>>>>> China Software Development Lab
>>>>> IBM
>>>>>
>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>>> For additional commands, e-mail: harmony-dev-
>> help@incubator.apache.org
>>>>>
>>>
>>> --
>>> Paulex Yang
>>> China Software Development Lab
>>> IBM
>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>
>>>
> 
> 
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> 
> 


---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org


Mime
View raw message