Return-Path: Delivered-To: apmail-incubator-harmony-dev-archive@www.apache.org Received: (qmail 51797 invoked from network); 14 Jun 2006 00:07:15 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (209.237.227.199) by minotaur.apache.org with SMTP; 14 Jun 2006 00:07:15 -0000 Received: (qmail 2049 invoked by uid 500); 14 Jun 2006 00:07:13 -0000 Delivered-To: apmail-incubator-harmony-dev-archive@incubator.apache.org Received: (qmail 2002 invoked by uid 500); 14 Jun 2006 00:07:12 -0000 Mailing-List: contact harmony-dev-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: harmony-dev@incubator.apache.org Delivered-To: mailing list harmony-dev@incubator.apache.org Received: (qmail 1989 invoked by uid 99); 14 Jun 2006 00:07:12 -0000 Received: from asf.osuosl.org (HELO asf.osuosl.org) (140.211.166.49) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 13 Jun 2006 17:07:12 -0700 X-ASF-Spam-Status: No, hits=0.9 required=10.0 tests=DNS_FROM_RFC_WHOIS,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (asf.osuosl.org: domain of nbeyer@kc.rr.com designates 24.94.166.115 as permitted sender) Received: from [24.94.166.115] (HELO ms-smtp-01.rdc-kc.rr.com) (24.94.166.115) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 13 Jun 2006 17:07:11 -0700 Received: from LITTLEGUY (CPE-24-31-248-236.kc.res.rr.com [24.31.248.236]) by ms-smtp-01.rdc-kc.rr.com (8.13.6/8.13.6) with ESMTP id k5E06moW007495 for ; Tue, 13 Jun 2006 19:06:48 -0500 (CDT) From: "Nathan Beyer" To: Subject: RE: [classlib] [testing] Coverage (was Re: 37% of total test execution time is spent in a single test) Date: Tue, 13 Jun 2006 19:06:48 -0500 Message-ID: <003301c68f46$6e3e7ef0$0f01a8c0@LITTLEGUY> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook 11 In-reply-to: <7273946b0606122236w2e40b9f8l75bdf15944da37f5@mail.gmail.com> X-MIMEOLE: Produced By Microsoft MimeOLE V6.00.2900.2869 Thread-index: AcaOq2AhUnLDftzWS0Gwm8p2Tw9CXwAmrelw X-Virus-Scanned: Symantec AntiVirus Scan Engine X-Virus-Checked: Checked by ClamAV on apache.org X-Spam-Rating: minotaur.apache.org 1.6.2 0/1000/N Is there any possibility of adding this to an Ant script that can optionally be run with the build scripts? I'd like to make this easier for everyone to run while developing. Has anyone tried any other coverage tools? Like TPTP for Eclipse, Clover or Corbetura (sp)? -Nathan > -----Original Message----- > From: Vladimir Ivanov [mailto:ivavladimir@gmail.com] > > Latest Harmony API source coverage by Harmony API unit tests results I > stored at wiki page > > http://wiki.apache.org/harmony/Coverage_information > > > > I'm going to refresh it bi-weekly (seems, it is enough for coverage). > > > > >I think we have got a agreement on the test naming convention[1], but > >for sure, there have been many (legacy) test cases before this > >agreement, and I think the volunteer is highly welcome to provide patch > >for them. > > > >[1]http://incubator.apache.org/harmony/subcomponents/classlibrary/testing > .html > > > > > If nobody objects I'm going to look through the unit tests to correct > package names according to the agreement (where needed). > > > > Thanks, > > Vladimir > > > > On 6/8/06, Paulex Yang wrote: > > > > Vladimir, > > > > Vladimir Ivanov wrote: > > > Thanks Paulex! > > > > > > I did the same, but could not send results due to spam filter J > > > Observations: > > > > > > 1. Coverage results look pretty much similar. > > > 2. Exclude list looks pretty much similar too, but, looks like it > > > depends on the way of data collection (I didn't run ant task and the > > > list is > > > a little bit different). > > Great. > > > > > > In any case, I think, when we run harmony on another VM exclude list > > will > > > have to be updated. > > > > > > > > > > > > May be we can start publishing the coverage information on wiki pages > > and > > > provide some updates time to time (I can do it)? > > > > > +1, and of course, you can only if no one in the mailing list objects > > and you'll have my welcome, and I think it will be even greater if these > > reports can be generated regularly like what JAPI is doing:) > > > > > > > > > One note: > > > > > > I noticed that different unit tests have very different package names > > > > > > Now the directory with all built tests copied to one place looks like: > > I think we have got a agreement on the test naming convention[1], but > > for sure, there have been many (legacy) test cases before this > > agreement, and I think the volunteer is highly welcome to provide patch > > for them. > > > > > [1]http://incubator.apache.org/harmony/subcomponents/classlibrary/testing. > html > > > > > > > > > > > > > > C:\coverage\tests\test>ls > > > > > > GZIPOutClose2.txt api config javax > > > tests > > > > > > GZIPOutFinish.txt api.injected dazzle org > xml > > > > > > > > GZIPOutWrite.txt binary gif prefs > > > > > > Inet6Address.golden.ser bundles impl serialization > > > > > > JDK2-3gabba.zip com impl.injected test.txt > > > > > > > > > > > > I think, it would be good if tests had unified package names. > > > > > > Why? - so far, just common sense, just to have an order in test suite > > > > > > Organization (if consider all unit tests as solid test suite). > > > > > > Thanks, > > > Vladimir > > > > > > For example, my exclude list for java.io is: > > > -java.io.BufferedInputStream , > > > -java.io.BufferedOutputStream, > > > -java.io.File , > > > -java.io.FileChannelFactory, > > > -java.io.FileDescriptor, > > > -java.io.FileInputStream, > > > -java.io.FileOutputStream, > > > -java.io.FilterInputStream , > > > -java.io.FilterOutputStream, > > > - java.io.InputStream, > > > -java.io.OutputStream, > > > -java.io.ObjectStreamField, > > > -java.io.PrintStream > > > > > > > > > On 6/6/06, Paulex Yang < paulex.yang@gmail.com > wrote: > > >> > > >> I've attach the scripts and excluded class lists to JIRA, please > refer > > >> to https://issues.apache.org/jira/browse/HARMONY-564 . Enjoy it:). > > >> > > >> Mark Hindess wrote: > > >> > On 2 June 2006 at 10:37, Paulex Yang < paulex.yang@gmail.com> > wrote: > > >> > > > >> >> Mark, > > >> >> > > >> >> I'm glad that there is someone else has interest on emma, I've > > >> tried it > > >> >> before. AFAIK, emma works by instrumentation, but sometimes for > > >> classes > > >> >> in bootclasspath, the instrumentation cannot work, there are two > > >> cases: > > >> > > >> >> 1. Some instrumented classes cannot be loaded by VM. > > >> >> 2. Some classes cannot be instrumented > > >> >> > > >> >> I have tried to look more inside to find some way to work around > > >> but I > > >> >> haven't got enough time yet. > > >> >> > > >> >> Specifically for nio-channel module, I had a list for these two > > cases > > >> (I > > >> >> believe the data is a little outdated and should be reevaluated) > > >> >> case 1. > > >> >> BaseByteBuffer.class > > >> >> Buffer.class > > >> >> BufferFactory.class > > >> >> ByteBuffer.class > > >> >> CharArrayBuffer.class > > >> >> CharBuffer.class > > >> >> HeapByteBuffer.class > > >> >> ReadWriteCharArrayBuffer.class > > >> >> ReadWriteHeapByteBuffer.class > > >> >> FileChannel.class > > >> >> AbstractInterruptibleChannel.class > > >> >> FileChannelImpl.class > > >> >> WriteOnlyFileChannel.class > > >> >> LockManager.class > > >> >> LockManager$1.class > > >> >> ReadOnlyFileChannel.class > > >> >> > > >> >> case 2: > > >> >> ByteChannel.class > > >> >> Channel.class > > >> >> GatheringByteChannel.class > > >> >> InterruptibleChannel.class > > >> >> WritableByteChannel.class > > >> >> > > >> >> And I have got some ant script and more excluded list for emma, if > > >> >> anyone has interests, I can upload it to JIRA. > > >> >> > > >> > > > >> > Yes! > > >> > > > >> > -Mark. > > >> > > > >> > > > >> >> Mark Hindess wrote: > > >> >> > > >> >>> Anyone tried using emma (emma.sf.net) to look at test coverage > > >> for our > > >> >>> tests? That might help ensure we don't trim them down too much? > > >> >>> > > >> >>> I suspect we will soon reach a point where there are > significantly > > >> fewer > > >> >>> obvious opportunities for small contributions - e.g. > > >> 1.5generification, > > >> >>> etc from looking at JAPI reports. Having test coverage reports > > >> might > > >> >>> help by highlighting areas for improvement. > > >> >>> > > >> >>> I had a quick try with: > > >> >>> > > >> >>> ant -f make/build.xml > > >> >>> ant -f modules/nio/make/build.xml test > > >> >>> java -cp emma.jar emma instr -m overwrite -cp > > >> deploy/jdk/jre/lib/boot/nio > > >> >>> > > >> >> .jar > > >> >> > > >> >>> and then: > > >> >>> > > >> >>> java -Xbootclasspath/p: emma.jar \ > > >> >>> -cp build/tests:modules/nio/bin/test:junit.jar:. \ > > >> >>> junit.textui.TestRunner \ > > >> >>> > org.apache.harmony.tests.java.nio.channels.SocketChannelTest > > >> >>> > > >> >>> (Running all the tests is a little tricky because test depends on > > >> build, > > >> >>> which depends on clean which wipes out the instr'd jar. ;-( ) > > >> >>> > > >> >>> Sadly it dies horribly with the stack trace below. I'll take > > >> another > > >> >>> look when I have a little more time, but I thought I'd mention it > > in > > >> >>> case anyone else had any thoughts on the subject. > > >> >>> > > >> >>> Regards, > > >> >>> > > >> >>> Mark. > > >> >>> > > >> >>> Exception in thread "main" java/lang/ExceptionInInitializerError > > >> >>> at java/lang/J9VMInternals.initialize > > >> (J9VMInternals.java:203) > > >> >>> at java/nio/Buffer.$VRi ( Buffer.java:4294967295) > > >> >>> at java/nio/Buffer. ( Buffer.java:4294967295) > > >> >>> at java/lang/J9VMInternals.initializeImpl (Native Method) > > >> >>> at java/lang/J9VMInternals.initialize ( > J9VMInternals.java > > >> :185) > > >> >>> at java/lang/J9VMInternals.initialize > > >> (J9VMInternals.java:154) > > >> >>> at java/lang/String.getBytes (String.java:783) > > >> >>> at org/apache/harmony/luni/util/Util.getBytes ( > > >> Util.java:54) > > >> >>> at java/lang/ClassLoader.loadLibraryWithPath > > >> (ClassLoader.java > > >> :998) > > >> >>> at java/lang/ClassLoader.loadLibraryWithClassLoader ( > > >> ClassLoader.ja > > >> >>> > > >> >> va:970) > > >> >> > > >> >>> at java/lang/System.loadLibrary (System.java:706) > > >> >>> at com/ibm/icu4jni/common/ICU4JNILoader.loadLibrary > > (Unknown > > >> Source > > >> >>> > > >> >> ) > > >> >> > > >> >>> at com/ibm/icu4jni/converters/NativeConverter. > > >> (Unknown Sou > > >> >>> > > >> >> rce) > > >> >> > > >> >>> at java/lang/J9VMInternals.initializeImpl (Native Method) > > >> >>> at java/lang/J9VMInternals.initialize > > >> ( J9VMInternals.java:185) > > >> >>> at > > com/ibm/icu4jni/charset/CharsetProviderICU.charsetForName > > >> (Unkno > > >> >>> > > >> >> wn Source) > > >> >> > > >> >>> at java/nio/charset/Charset.forNameInternal > > >> (Charset.java:450) > > >> >>> at java/nio/charset/Charset.forName (Charset.java:513) > > >> >>> at java/lang/String.defaultCharset ( String.java:688) > > >> >>> at java/lang/String. ( String.java:252) > > >> >>> at org/apache/harmony/luni/util/Util.toString (Util.java > > :74) > > >> >>> at java/lang/System.getPropertyList (Native Method) > > >> >>> at java/lang/System.ensureProperties ( System.java:496) > > >> >>> at java/lang/System. (System.java:55) > > >> >>> at java/lang/J9VMInternals.initializeImpl (Native Method) > > >> >>> at java/lang/J9VMInternals.initialize > > >> (J9VMInternals.java:185) > > >> >>> at java/lang/ClassLoader.initializeClassLoaders ( > > >> ClassLoader.java:6 > > >> >>> > > >> >> 5) > > >> >> > > >> >>> at java/lang/Thread.initialize (Thread.java:301) > > >> >>> at java/lang/Thread. (Thread.java:120) > > >> >>> java/lang/NullPointerException > > >> >>> at java/lang/Throwable.printStackTrace ( Throwable.java > > :231) > > >> >>> at > com/vladium/emma/rt/RTExitHook.createClassLoaderClosure > > >> (RTExitH > > >> >>> > > >> >> ook.java:63) > > >> >> > > >> >>> at com/vladium/emma/rt/RT.reset ( RT.java:92) > > >> >>> at com/vladium/emma/rt/RT. (RT.java:238) > > >> >>> at java/lang/J9VMInternals.initializeImpl (Native Method) > > >> >>> at java/lang/J9VMInternals.initialize ( > J9VMInternals.java > > >> :185) > > >> >>> at java/nio/Buffer.$VRi ( Buffer.java:4294967295) > > >> >>> at java/nio/Buffer. (Buffer.java:4294967295) > > >> >>> at java/lang/J9VMInternals.initializeImpl (Native Method) > > >> >>> at java/lang/J9VMInternals.initialize > > >> (J9VMInternals.java:185) > > >> >>> at java/lang/J9VMInternals.initialize > > >> (J9VMInternals.java:154) > > >> >>> at java/lang/String.getBytes ( String.java :783) > > >> >>> at org/apache/harmony/luni/util/Util.getBytes (Util.java > > :54) > > >> >>> at java/lang/ClassLoader.loadLibraryWithPath > > >> (ClassLoader.java > > >> :998) > > >> >>> at java/lang/ClassLoader.loadLibraryWithClassLoader ( > > >> ClassLoader.ja > > >> >>> > > >> >> va:970) > > >> >> > > >> >>> at java/lang/System.loadLibrary ( System.java:706) > > >> >>> at com/ibm/icu4jni/common/ICU4JNILoader.loadLibrary > > (Unknown > > >> Source > > >> >>> > > >> >> ) > > >> >> > > >> >>> at com/ibm/icu4jni/converters/NativeConverter. > > >> (Unknown Sou > > >> >>> > > >> >> rce) > > >> >> > > >> >>> at java/lang/J9VMInternals.initializeImpl (Native Method) > > >> >>> at java/lang/J9VMInternals.initialize > > >> (J9VMInternals.java:185) > > >> >>> at > > com/ibm/icu4jni/charset/CharsetProviderICU.charsetForName > > >> (Unkno > > >> >>> > > >> >> wn Source) > > >> >> > > >> >>> at java/nio/charset/Charset.forNameInternal > > >> (Charset.java:450) > > >> >>> at java/nio/charset/Charset.forName (Charset.java:513) > > >> >>> at java/lang/String.defaultCharset ( String.java:688) > > >> >>> at java/lang/String. (String.java:252) > > >> >>> at org/apache/harmony/luni/util/Util.toString (Util.java > > :74) > > >> >>> at java/lang/System.getPropertyList (Native Method) > > >> >>> at java/lang/System.ensureProperties (System.java:496) > > >> >>> at java/lang/System. (System.java:55) > > >> >>> at java/lang/J9VMInternals.initializeImpl (Native Method) > > >> >>> at java/lang/J9VMInternals.initialize > > >> (J9VMInternals.java:185) > > >> >>> at java/lang/ClassLoader.initializeClassLoaders ( > > >> ClassLoader.java:6 > > >> >>> > > >> >> 5) > > >> >> > > >> >>> at java/lang/Thread.initialize (Thread.java:301) > > >> >>> at java/lang/Thread. (Thread.java :120) > > >> >>> JVMJ9VM015W Initialization error for library jclclear_23(14): > > >> JVMJ9VM009E J > > >> >>> > > >> >> 9VMDllMain failed > > >> >> > > >> >>> HMYEXEL062E Internal VM error: Failed to create Java VM > > >> >>> FAILED. > > >> >>> > > >> >>> > > >> >>> On 1 June 2006 at 22:26, "Mikhail Loenko" < mloenko@gmail.com > > > >> wrote: > > >> >>> > > >> >>> > > >> >>>> 2006/6/1, Geir Magnusson Jr < geir@pobox.com>: > > >> >>>> > > >> >>>> > > >> >>>>> Mikhail Loenko wrote: > > >> >>>>> > > >> >>>>> > > >> >>>>>> 2006/6/1, Geir Magnusson Jr < geir@pobox.com >: > > >> >>>>>> > > >> >>>>>> > > >> >>>>>>> I don't quite grok what you mean - could you post what you > are > > >> >>>>>>> suggesting in code here on -dev? > > >> >>>>>>> > > >> >>>>>>> > > >> >>>>>> Index: > > >> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.ja > > >> >>>>>> > > >> >> v > > >> >> > > >> >>>>>> > > >> >>>>>> > > >> >>>> a > > >> >>>> > > >> >>>> > > >> >>>>>> > > >> =================================================================== > > >> >>>>>> --- > > >> >>>>>> > > >> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.java > > >> >>>>>> (revision > > >> >>>>>> 410480) > > >> >>>>>> +++ > > >> >>>>>> > > >> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.java > > >> > > >> >>>>>> (working > > >> >>>>>> copy) > > >> >>>>>> @@ -785,7 +785,7 @@ > > >> >>>>>> "FFFF:FFFF:FFFF:FFFF:FFFF:FFFF: > 255.255.255.255 > > ", > > >> >>>>>> "0:0:0:0:0:0:0:0", "0:0:0:0:0:0: 0.0.0.0" }; > > >> >>>>>> > > >> >>>>>> - String invalidIPAddresses[] = { ":", "FFFF:FFFF", > > "1:1", > > >> >>>>>> "::1.2.3.444" }; > > >> >>>>>> + String invalidIPAddresses[] = { "FFFF:FFFF" }; > > >> >>>>>> > > >> >>>>>> for (int i = 0; i < validIPAddresses.length; i++) { > > >> >>>>>> > > >> >>>>>> > > >> >>>>>> > > >> >>>>>> > > >> >>>>> Thx - why do you want to remove testing the others? Because > you > > >> believe > > >> >>>>> that it is in effect testing the resolver in the OS since each > > >> returns > > >> >>>>> the same answer,and thus it's testing the same thing over and > > >> over? > > >> >>>>> > > >> >>>>> > > >> >>>> I believe so, but I'd like to hear confirmation from code > authors > > >> >>>> > > >> >>>> Thanks, > > >> >>>> Mikhail > > >> >>>> > > >> >>>> > > >> >>> > > >> >>> > > >> --------------------------------------------------------------------- > > >> >>> Terms of use : http://incubator.apache.org/harmony/mailing.html > > >> >>> To unsubscribe, e-mail: harmony-dev- > unsubscribe@incubator.apache.org > > > > >> >>> For additional commands, e-mail: > > >> harmony-dev-help@incubator.apache.org > > >> >>> > > >> >>> > > >> >>> > > >> >>> > > >> >> -- > > >> >> Paulex Yang > > >> >> China Software Development Lab > > >> >> IBM > > >> >> > > >> >> > > >> >> > > >> >> > > --------------------------------------------------------------------- > > >> >> Terms of use : http://incubator.apache.org/harmony/mailing.html > > >> >> To unsubscribe, e-mail: harmony-dev- > unsubscribe@incubator.apache.org > > >> >> For additional commands, e-mail: > > >> harmony-dev-help@incubator.apache.org > > >> >> > > >> > > > >> > > > >> > > > >> > ------------------------------------------------------------------- > -- > > > > >> > Terms of use : http://incubator.apache.org/harmony/mailing.html > > >> > To unsubscribe, e-mail: harmony-dev- > unsubscribe@incubator.apache.org > > >> > For additional commands, e-mail: > > harmony-dev-help@incubator.apache.org > > >> > > > >> > > > >> > > > >> > > >> > > >> -- > > >> Paulex Yang > > >> China Software Development Lab > > >> IBM > > >> > > >> > > >> > > >> --------------------------------------------------------------------- > > >> Terms of use : http://incubator.apache.org/harmony/mailing.html > > >> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org > > >> For additional commands, e-mail: harmony-dev- > help@incubator.apache.org > > >> > > >> > > > > > > > > > -- > > Paulex Yang > > China Software Development Lab > > IBM > > > > > > > > --------------------------------------------------------------------- > > Terms of use : http://incubator.apache.org/harmony/mailing.html > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org > > For additional commands, e-mail: harmony-dev-help@incubator.apache.org > > > > --------------------------------------------------------------------- Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org For additional commands, e-mail: harmony-dev-help@incubator.apache.org