harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paulex Yang <paulex.y...@gmail.com>
Subject Re: [classlib] [testing] Coverage (was Re: 37% of total test execution time is spent in a single test)
Date Thu, 08 Jun 2006 05:09:22 GMT
Vladimir,

Vladimir Ivanov wrote:
> Thanks Paulex!
>
> I did the same, but could not send results due to spam filter J
> Observations:
>
>   1. Coverage results look pretty much similar.
>   2. Exclude list looks pretty much similar too, but, looks like it
>   depends on the way of data collection (I didn't run ant task and the 
> list is
>   a little bit different).
Great.
>
> In any case, I think, when we run harmony on another VM exclude list will
> have to be updated.
>
>
>
> May be we can start publishing the coverage information on wiki pages and
> provide some updates time to time (I can do it)?
>
+1,  and of course, you can only if  no one in the mailing list objects 
and you'll have my welcome, and I think it will be even greater if these 
reports can be generated regularly like what JAPI is doing:)
>
>
> One note:
>
> I noticed that different unit tests have very different package names
>
> Now the directory with all built tests copied to one place looks like:
I think we have got a agreement on the test naming convention[1], but 
for sure, there have been many (legacy) test cases before this 
agreement, and I think the volunteer is highly welcome to provide patch 
for them.

[1]http://incubator.apache.org/harmony/subcomponents/classlibrary/testing.html
>
>
>
> C:\coverage\tests\test>ls
>
> GZIPOutClose2.txt        api           config         javax          
> tests
>
> GZIPOutFinish.txt        api.injected  dazzle         org            xml
>
> GZIPOutWrite.txt         binary        gif            prefs
>
> Inet6Address.golden.ser  bundles       impl           serialization
>
> JDK2-3gabba.zip          com           impl.injected  test.txt
>
>
>
> I think, it would be good if tests had unified package names.
>
> Why? – so far, just common sense, just to have an order in test suite
>
> Organization (if consider all unit tests as solid test suite).
>
> Thanks,
>  Vladimir
>
> For example, my exclude list for java.io is:
> -java.io.BufferedInputStream,
> -java.io.BufferedOutputStream,
> -java.io.File,
> -java.io.FileChannelFactory,
> -java.io.FileDescriptor,
> -java.io.FileInputStream,
> -java.io.FileOutputStream,
> -java.io.FilterInputStream ,
> -java.io.FilterOutputStream,
> -java.io.InputStream,
> -java.io.OutputStream,
> -java.io.ObjectStreamField,
> -java.io.PrintStream
>
>
> On 6/6/06, Paulex Yang <paulex.yang@gmail.com> wrote:
>>
>> I've attach the scripts and excluded class lists to JIRA,  please refer
>> to https://issues.apache.org/jira/browse/HARMONY-564.  Enjoy it:).
>>
>> Mark Hindess wrote:
>> > On 2 June 2006 at 10:37, Paulex Yang <paulex.yang@gmail.com> wrote:
>> >
>> >> Mark,
>> >>
>> >> I'm glad that there is someone else has interest on emma, I've 
>> tried it
>> >> before. AFAIK, emma works by instrumentation, but sometimes for 
>> classes
>> >> in bootclasspath, the instrumentation cannot work, there are two 
>> cases:
>>
>> >> 1. Some instrumented classes cannot be loaded by VM.
>> >> 2. Some classes cannot be instrumented
>> >>
>> >> I have tried to look more inside to find some way to work around 
>> but I
>> >> haven't got enough time yet.
>> >>
>> >> Specifically for nio-channel module, I had a list for these two cases
>> (I
>> >> believe the data is a little outdated and should be reevaluated)
>> >> case 1.
>> >> BaseByteBuffer.class
>> >> Buffer.class
>> >> BufferFactory.class
>> >> ByteBuffer.class
>> >> CharArrayBuffer.class
>> >> CharBuffer.class
>> >> HeapByteBuffer.class
>> >> ReadWriteCharArrayBuffer.class
>> >> ReadWriteHeapByteBuffer.class
>> >> FileChannel.class
>> >> AbstractInterruptibleChannel.class
>> >> FileChannelImpl.class
>> >> WriteOnlyFileChannel.class
>> >> LockManager.class
>> >> LockManager$1.class
>> >> ReadOnlyFileChannel.class
>> >>
>> >> case 2:
>> >> ByteChannel.class
>> >> Channel.class
>> >> GatheringByteChannel.class
>> >> InterruptibleChannel.class
>> >> WritableByteChannel.class
>> >>
>> >> And I have got some ant script and more excluded list for emma, if
>> >> anyone has interests, I can upload it to JIRA.
>> >>
>> >
>> > Yes!
>> >
>> > -Mark.
>> >
>> >
>> >> Mark Hindess wrote:
>> >>
>> >>> Anyone tried using emma (emma.sf.net) to look at test coverage 
>> for our
>> >>> tests?  That might help ensure we don't trim them down too much?
>> >>>
>> >>> I suspect we will soon reach a point where there are significantly
>> fewer
>> >>> obvious opportunities for small contributions - e.g. 
>> 1.5generification,
>> >>> etc from looking at JAPI reports.  Having test coverage reports 
>> might
>> >>> help by highlighting areas for improvement.
>> >>>
>> >>> I had a quick try with:
>> >>>
>> >>>   ant -f make/build.xml
>> >>>   ant -f modules/nio/make/build.xml test
>> >>>   java -cp emma.jar emma instr -m overwrite -cp
>> deploy/jdk/jre/lib/boot/nio
>> >>>
>> >> .jar
>> >>
>> >>> and then:
>> >>>
>> >>>   java -Xbootclasspath/p: emma.jar \
>> >>>        -cp build/tests:modules/nio/bin/test:junit.jar:. \
>> >>>        junit.textui.TestRunner \
>> >>>        org.apache.harmony.tests.java.nio.channels.SocketChannelTest
>> >>>
>> >>> (Running all the tests is a little tricky because test depends on
>> build,
>> >>> which depends on clean which wipes out the instr'd jar. ;-( )
>> >>>
>> >>> Sadly it dies horribly with the stack trace below.  I'll take 
>> another
>> >>> look when I have a little more time, but I thought I'd mention it in
>> >>> case anyone else had any thoughts on the subject.
>> >>>
>> >>> Regards,
>> >>>
>> >>> Mark.
>> >>>
>> >>> Exception in thread "main" java/lang/ExceptionInInitializerError
>> >>>         at java/lang/J9VMInternals.initialize 
>> (J9VMInternals.java:203)
>> >>>         at java/nio/Buffer.$VRi ( Buffer.java:4294967295)
>> >>>         at java/nio/Buffer.<clinit> (Buffer.java:4294967295)
>> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
>> >>>         at java/lang/J9VMInternals.initialize ( J9VMInternals.java
>> :185)
>> >>>         at java/lang/J9VMInternals.initialize 
>> (J9VMInternals.java:154)
>> >>>         at java/lang/String.getBytes (String.java:783)
>> >>>         at org/apache/harmony/luni/util/Util.getBytes ( 
>> Util.java:54)
>> >>>         at java/lang/ClassLoader.loadLibraryWithPath 
>> (ClassLoader.java
>> :998)
>> >>>         at java/lang/ClassLoader.loadLibraryWithClassLoader (
>> ClassLoader.ja
>> >>>
>> >> va:970)
>> >>
>> >>>         at java/lang/System.loadLibrary (System.java:706)
>> >>>         at com/ibm/icu4jni/common/ICU4JNILoader.loadLibrary (Unknown
>> Source
>> >>>
>> >> )
>> >>
>> >>>         at com/ibm/icu4jni/converters/NativeConverter.<clinit>
>> (Unknown Sou
>> >>>
>> >> rce)
>> >>
>> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
>> >>>         at java/lang/J9VMInternals.initialize 
>> (J9VMInternals.java:185)
>> >>>         at com/ibm/icu4jni/charset/CharsetProviderICU.charsetForName
>> (Unkno
>> >>>
>> >> wn Source)
>> >>
>> >>>         at java/nio/charset/Charset.forNameInternal 
>> (Charset.java:450)
>> >>>         at java/nio/charset/Charset.forName (Charset.java:513)
>> >>>         at java/lang/String.defaultCharset ( String.java:688)
>> >>>         at java/lang/String.<init> (String.java:252)
>> >>>         at org/apache/harmony/luni/util/Util.toString (Util.java:74)
>> >>>         at java/lang/System.getPropertyList (Native Method)
>> >>>         at java/lang/System.ensureProperties (System.java:496)
>> >>>         at java/lang/System.<clinit> (System.java:55)
>> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
>> >>>         at java/lang/J9VMInternals.initialize 
>> (J9VMInternals.java:185)
>> >>>         at java/lang/ClassLoader.initializeClassLoaders (
>> ClassLoader.java:6
>> >>>
>> >> 5)
>> >>
>> >>>         at java/lang/Thread.initialize (Thread.java:301)
>> >>>         at java/lang/Thread.<init> (Thread.java:120)
>> >>> java/lang/NullPointerException
>> >>>         at java/lang/Throwable.printStackTrace ( Throwable.java:231)
>> >>>         at com/vladium/emma/rt/RTExitHook.createClassLoaderClosure
>> (RTExitH
>> >>>
>> >> ook.java:63)
>> >>
>> >>>         at com/vladium/emma/rt/RT.reset ( RT.java:92)
>> >>>         at com/vladium/emma/rt/RT.<clinit> (RT.java:238)
>> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
>> >>>         at java/lang/J9VMInternals.initialize ( J9VMInternals.java
>> :185)
>> >>>         at java/nio/Buffer.$VRi (Buffer.java:4294967295)
>> >>>         at java/nio/Buffer.<clinit> (Buffer.java:4294967295)
>> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
>> >>>         at java/lang/J9VMInternals.initialize 
>> (J9VMInternals.java:185)
>> >>>         at java/lang/J9VMInternals.initialize 
>> (J9VMInternals.java:154)
>> >>>         at java/lang/String.getBytes ( String.java:783)
>> >>>         at org/apache/harmony/luni/util/Util.getBytes (Util.java:54)
>> >>>         at java/lang/ClassLoader.loadLibraryWithPath 
>> (ClassLoader.java
>> :998)
>> >>>         at java/lang/ClassLoader.loadLibraryWithClassLoader (
>> ClassLoader.ja
>> >>>
>> >> va:970)
>> >>
>> >>>         at java/lang/System.loadLibrary (System.java:706)
>> >>>         at com/ibm/icu4jni/common/ICU4JNILoader.loadLibrary (Unknown
>> Source
>> >>>
>> >> )
>> >>
>> >>>         at com/ibm/icu4jni/converters/NativeConverter.<clinit>
>> (Unknown Sou
>> >>>
>> >> rce)
>> >>
>> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
>> >>>         at java/lang/J9VMInternals.initialize 
>> (J9VMInternals.java:185)
>> >>>         at com/ibm/icu4jni/charset/CharsetProviderICU.charsetForName
>> (Unkno
>> >>>
>> >> wn Source)
>> >>
>> >>>         at java/nio/charset/Charset.forNameInternal 
>> (Charset.java:450)
>> >>>         at java/nio/charset/Charset.forName (Charset.java:513)
>> >>>         at java/lang/String.defaultCharset ( String.java:688)
>> >>>         at java/lang/String.<init> (String.java:252)
>> >>>         at org/apache/harmony/luni/util/Util.toString (Util.java:74)
>> >>>         at java/lang/System.getPropertyList (Native Method)
>> >>>         at java/lang/System.ensureProperties (System.java:496)
>> >>>         at java/lang/System.<clinit> (System.java:55)
>> >>>         at java/lang/J9VMInternals.initializeImpl (Native Method)
>> >>>         at java/lang/J9VMInternals.initialize 
>> (J9VMInternals.java:185)
>> >>>         at java/lang/ClassLoader.initializeClassLoaders (
>> ClassLoader.java:6
>> >>>
>> >> 5)
>> >>
>> >>>         at java/lang/Thread.initialize (Thread.java:301)
>> >>>         at java/lang/Thread.<init> (Thread.java:120)
>> >>> JVMJ9VM015W Initialization error for library jclclear_23(14):
>> JVMJ9VM009E J
>> >>>
>> >> 9VMDllMain failed
>> >>
>> >>> HMYEXEL062E Internal VM error: Failed to create Java VM
>> >>> FAILED.
>> >>>
>> >>>
>> >>> On 1 June 2006 at 22:26, "Mikhail Loenko" < mloenko@gmail.com>

>> wrote:
>> >>>
>> >>>
>> >>>> 2006/6/1, Geir Magnusson Jr <geir@pobox.com>:
>> >>>>
>> >>>>
>> >>>>> Mikhail Loenko wrote:
>> >>>>>
>> >>>>>
>> >>>>>> 2006/6/1, Geir Magnusson Jr <geir@pobox.com >:
>> >>>>>>
>> >>>>>>
>> >>>>>>> I don't quite grok what you mean - could you post what
you are
>> >>>>>>> suggesting in code here on -dev?
>> >>>>>>>
>> >>>>>>>
>> >>>>>> Index:
>> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.ja
>> >>>>>>
>> >> v
>> >>
>> >>>>>>
>> >>>>>>
>> >>>> a
>> >>>>
>> >>>>
>> >>>>>> 
>> ===================================================================
>> >>>>>> ---
>> >>>>>> 
>> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.java
>> >>>>>> (revision
>> >>>>>> 410480)
>> >>>>>> +++
>> >>>>>> 
>> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.java
>>
>> >>>>>> (working
>> >>>>>> copy)
>> >>>>>> @@ -785,7 +785,7 @@
>> >>>>>>                 "FFFF:FFFF:FFFF:FFFF:FFFF:FFFF: 255.255.255.255",
>> >>>>>>                 "0:0:0:0:0:0:0:0", "0:0:0:0:0:0:0.0.0.0"
};
>> >>>>>>
>> >>>>>> -        String invalidIPAddresses[] = { ":", "FFFF:FFFF",
"1:1",
>> >>>>>> "::1.2.3.444" };
>> >>>>>> +        String invalidIPAddresses[] = { "FFFF:FFFF" };
>> >>>>>>
>> >>>>>>         for (int i = 0; i < validIPAddresses.length;
i++) {
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>> Thx - why do you want to remove testing the others?  Because
you
>> believe
>> >>>>> that it is in effect testing the resolver in the OS since each
>> returns
>> >>>>> the same answer,and thus it's testing the same thing over and

>> over?
>> >>>>>
>> >>>>>
>> >>>> I believe so, but I'd like to hear confirmation from code authors
>> >>>>
>> >>>> Thanks,
>> >>>> Mikhail
>> >>>>
>> >>>>
>> >>>
>> >>> 
>> ---------------------------------------------------------------------
>> >>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>> >>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>> >>> For additional commands, e-mail: 
>> harmony-dev-help@incubator.apache.org
>> >>>
>> >>>
>> >>>
>> >>>
>> >> --
>> >> Paulex Yang
>> >> China Software Development Lab
>> >> IBM
>> >>
>> >>
>> >>
>> >> ---------------------------------------------------------------------
>> >> Terms of use : http://incubator.apache.org/harmony/mailing.html
>> >> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>> >> For additional commands, e-mail: 
>> harmony-dev-help@incubator.apache.org
>> >>
>> >
>> >
>> >
>> > ---------------------------------------------------------------------
>> > Terms of use : http://incubator.apache.org/harmony/mailing.html
>> > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>> > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>> >
>> >
>> >
>>
>>
>> -- 
>> Paulex Yang
>> China Software Development Lab
>> IBM
>>
>>
>>
>> ---------------------------------------------------------------------
>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>
>>
>


-- 
Paulex Yang
China Software Development Lab
IBM



---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org


Mime
View raw message