harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vladimir Ivanov" <ivavladi...@gmail.com>
Subject Re: [classlib] [testing] Coverage (was Re: 37% of total test execution time is spent in a single test)
Date Tue, 13 Jun 2006 10:38:19 GMT
The detailed info available at  http://viv.byethost15.com/ (this address
also pointed on wiki-page).
Do you need additional information ?

Thanks,
  Vladimir


On 6/13/06, Mikhail Loenko <mloenko@gmail.com> wrote:
>
> Great!
>
> Is it possible to see uncovered classes/methods/lines?
>
> Thanks,
> Mikhail
>
> P.S. I think java.awt does not currently belong to beans module,
> though who knows what the future may bring us :)
>
> 2006/6/13, Vladimir Ivanov <ivavladimir@gmail.com>:
> > Latest Harmony API source coverage by Harmony API unit tests results I
> > stored at wiki page
> >
> > http://wiki.apache.org/harmony/Coverage_information
> >
> >
> >
> > I'm going to refresh it bi-weekly (seems, it is enough for coverage).
> >
> >
> >
> > >I think we have got a agreement on the test naming convention[1], but
> > >for sure, there have been many (legacy) test cases before this
> > >agreement, and I think the volunteer is highly welcome to provide patch
> > >for them.
> > >
> >
> >[1]http://incubator.apache.org/harmony/subcomponents/classlibrary/testing.html
> >
> > >
> >
> > If nobody objects I'm going to look through the unit tests to correct
> > package names according to the agreement (where needed).
> >
> >
> >
> >  Thanks,
> >
> >   Vladimir
> >
> >
> >
> > On 6/8/06, Paulex Yang <paulex.yang@gmail.com > wrote:
> > >
> > > Vladimir,
> > >
> > > Vladimir Ivanov wrote:
> > > > Thanks Paulex!
> > > >
> > > > I did the same, but could not send results due to spam filter J
> > > > Observations:
> > > >
> > > >   1. Coverage results look pretty much similar.
> > > >   2. Exclude list looks pretty much similar too, but, looks like it
> > > >   depends on the way of data collection (I didn't run ant task and
> the
> > > > list is
> > > >   a little bit different).
> > > Great.
> > > >
> > > > In any case, I think, when we run harmony on another VM exclude list
> > > will
> > > > have to be updated.
> > > >
> > > >
> > > >
> > > > May be we can start publishing the coverage information on wiki
> pages
> > > and
> > > > provide some updates time to time (I can do it)?
> > > >
> > > +1,  and of course, you can only if  no one in the mailing list
> objects
> > > and you'll have my welcome, and I think it will be even greater if
> these
> > > reports can be generated regularly like what JAPI is doing:)
> > > >
> > > >
> > > > One note:
> > > >
> > > > I noticed that different unit tests have very different package
> names
> > > >
> > > > Now the directory with all built tests copied to one place looks
> like:
> > > I think we have got a agreement on the test naming convention[1], but
> > > for sure, there have been many (legacy) test cases before this
> > > agreement, and I think the volunteer is highly welcome to provide
> patch
> > > for them.
> > >
> > >
> [1]http://incubator.apache.org/harmony/subcomponents/classlibrary/testing.html
> > >
> > > >
> > > >
> > > >
> > > > C:\coverage\tests\test>ls
> > > >
> > > > GZIPOutClose2.txt        api           config         javax
> > > > tests
> > > >
> > > > GZIPOutFinish.txt        api.injected  dazzle
> org            xml
> > >
> > > >
> > > > GZIPOutWrite.txt         binary        gif            prefs
> > > >
> > > > Inet6Address.golden.ser  bundles       impl           serialization
> > > >
> > > > JDK2-3gabba.zip          com           impl.injected   test.txt
> > > >
> > > >
> > > >
> > > > I think, it would be good if tests had unified package names.
> > > >
> > > > Why? – so far, just common sense, just to have an order in test
> suite
> > > >
> > > > Organization (if consider all unit tests as solid test suite).
> > > >
> > > > Thanks,
> > > >  Vladimir
> > > >
> > > > For example, my exclude list for java.io is:
> > > > -java.io.BufferedInputStream ,
> > > > -java.io.BufferedOutputStream,
> > > > -java.io.File ,
> > > > -java.io.FileChannelFactory,
> > > > -java.io.FileDescriptor,
> > > > -java.io.FileInputStream,
> > > > -java.io.FileOutputStream,
> > > > -java.io.FilterInputStream ,
> > > > -java.io.FilterOutputStream,
> > > > - java.io.InputStream,
> > > > -java.io.OutputStream,
> > > > -java.io.ObjectStreamField,
> > > > -java.io.PrintStream
> > > >
> > > >
> > > > On 6/6/06, Paulex Yang < paulex.yang@gmail.com > wrote:
> > > >>
> > > >> I've attach the scripts and excluded class lists to JIRA,  please
> refer
> > > >> to https://issues.apache.org/jira/browse/HARMONY-564 .  Enjoy it:).
> > > >>
> > > >> Mark Hindess wrote:
> > > >> > On 2 June 2006 at 10:37, Paulex Yang < paulex.yang@gmail.com>
> wrote:
> > > >> >
> > > >> >> Mark,
> > > >> >>
> > > >> >> I'm glad that there is someone else has interest on emma,
I've
> > > >> tried it
> > > >> >> before. AFAIK, emma works by instrumentation, but sometimes
for
> > > >> classes
> > > >> >> in bootclasspath, the instrumentation cannot work, there
are two
> > > >> cases:
> > > >>
> > > >> >> 1. Some instrumented classes cannot be loaded by VM.
> > > >> >> 2. Some classes cannot be instrumented
> > > >> >>
> > > >> >> I have tried to look more inside to find some way to work
around
> > > >> but I
> > > >> >> haven't got enough time yet.
> > > >> >>
> > > >> >> Specifically for nio-channel module, I had a list for these
two
> > > cases
> > > >> (I
> > > >> >> believe the data is a little outdated and should be reevaluated)
> > > >> >> case 1.
> > > >> >> BaseByteBuffer.class
> > > >> >> Buffer.class
> > > >> >> BufferFactory.class
> > > >> >> ByteBuffer.class
> > > >> >> CharArrayBuffer.class
> > > >> >> CharBuffer.class
> > > >> >> HeapByteBuffer.class
> > > >> >> ReadWriteCharArrayBuffer.class
> > > >> >> ReadWriteHeapByteBuffer.class
> > > >> >> FileChannel.class
> > > >> >> AbstractInterruptibleChannel.class
> > > >> >> FileChannelImpl.class
> > > >> >> WriteOnlyFileChannel.class
> > > >> >> LockManager.class
> > > >> >> LockManager$1.class
> > > >> >> ReadOnlyFileChannel.class
> > > >> >>
> > > >> >> case 2:
> > > >> >> ByteChannel.class
> > > >> >> Channel.class
> > > >> >> GatheringByteChannel.class
> > > >> >> InterruptibleChannel.class
> > > >> >> WritableByteChannel.class
> > > >> >>
> > > >> >> And I have got some ant script and more excluded list for
emma,
> if
> > > >> >> anyone has interests, I can upload it to JIRA.
> > > >> >>
> > > >> >
> > > >> > Yes!
> > > >> >
> > > >> > -Mark.
> > > >> >
> > > >> >
> > > >> >> Mark Hindess wrote:
> > > >> >>
> > > >> >>> Anyone tried using emma (emma.sf.net) to look at test
coverage
> > > >> for our
> > > >> >>> tests?  That might help ensure we don't trim them down
too
> much?
> > > >> >>>
> > > >> >>> I suspect we will soon reach a point where there are
> significantly
> > > >> fewer
> > > >> >>> obvious opportunities for small contributions - e.g.
> > > >> 1.5generification,
> > > >> >>> etc from looking at JAPI reports.  Having test coverage
reports
> > > >> might
> > > >> >>> help by highlighting areas for improvement.
> > > >> >>>
> > > >> >>> I had a quick try with:
> > > >> >>>
> > > >> >>>   ant -f make/build.xml
> > > >> >>>   ant -f modules/nio/make/build.xml test
> > > >> >>>   java -cp emma.jar emma instr -m overwrite -cp
> > > >> deploy/jdk/jre/lib/boot/nio
> > > >> >>>
> > > >> >> .jar
> > > >> >>
> > > >> >>> and then:
> > > >> >>>
> > > >> >>>   java -Xbootclasspath/p: emma.jar \
> > > >> >>>        -cp build/tests:modules/nio/bin/test:junit.jar:.
\
> > > >> >>>        junit.textui.TestRunner \
> > > >> >>>
> org.apache.harmony.tests.java.nio.channels.SocketChannelTest
> > > >> >>>
> > > >> >>> (Running all the tests is a little tricky because test
depends
> on
> > > >> build,
> > > >> >>> which depends on clean which wipes out the instr'd jar.
;-( )
> > > >> >>>
> > > >> >>> Sadly it dies horribly with the stack trace below.  I'll
take
> > > >> another
> > > >> >>> look when I have a little more time, but I thought I'd
mention
> it
> > > in
> > > >> >>> case anyone else had any thoughts on the subject.
> > > >> >>>
> > > >> >>> Regards,
> > > >> >>>
> > > >> >>> Mark.
> > > >> >>>
> > > >> >>> Exception in thread "main"
> java/lang/ExceptionInInitializerError
> > > >> >>>         at java/lang/J9VMInternals.initialize
> > > >> (J9VMInternals.java:203)
> > > >> >>>         at java/nio/Buffer.$VRi ( Buffer.java:4294967295)
> > > >> >>>         at java/nio/Buffer.<clinit> ( Buffer.java:4294967295)
> > > >> >>>         at java/lang/J9VMInternals.initializeImpl (Native
> Method)
> > > >> >>>         at java/lang/J9VMInternals.initialize (
> J9VMInternals.java
> > > >> :185)
> > > >> >>>         at java/lang/J9VMInternals.initialize
> > > >> (J9VMInternals.java:154)
> > > >> >>>         at java/lang/String.getBytes (String.java:783)
> > > >> >>>         at org/apache/harmony/luni/util/Util.getBytes
(
> > > >> Util.java:54)
> > > >> >>>         at java/lang/ClassLoader.loadLibraryWithPath
> > > >> (ClassLoader.java
> > > >> :998)
> > > >> >>>         at java/lang/ClassLoader.loadLibraryWithClassLoader
(
> > > >> ClassLoader.ja
> > > >> >>>
> > > >> >> va:970)
> > > >> >>
> > > >> >>>         at java/lang/System.loadLibrary (System.java:706)
> > > >> >>>         at com/ibm/icu4jni/common/ICU4JNILoader.loadLibrary
> > > (Unknown
> > > >> Source
> > > >> >>>
> > > >> >> )
> > > >> >>
> > > >> >>>         at com/ibm/icu4jni/converters/NativeConverter.<clinit>
> > > >> (Unknown Sou
> > > >> >>>
> > > >> >> rce)
> > > >> >>
> > > >> >>>         at java/lang/J9VMInternals.initializeImpl (Native
> Method)
> > > >> >>>         at java/lang/J9VMInternals.initialize
> > > >> ( J9VMInternals.java:185)
> > > >> >>>         at
> > > com/ibm/icu4jni/charset/CharsetProviderICU.charsetForName
> > > >> (Unkno
> > > >> >>>
> > > >> >> wn Source)
> > > >> >>
> > > >> >>>         at java/nio/charset/Charset.forNameInternal
> > > >> (Charset.java:450)
> > > >> >>>         at java/nio/charset/Charset.forName (Charset.java:513)
> > > >> >>>         at java/lang/String.defaultCharset ( String.java:688)
> > > >> >>>         at java/lang/String.<init> ( String.java:252)
> > > >> >>>         at org/apache/harmony/luni/util/Util.toString
(
> Util.java
> > > :74)
> > > >> >>>         at java/lang/System.getPropertyList (Native Method)
> > > >> >>>         at java/lang/System.ensureProperties ( System.java:496)
> > > >> >>>         at java/lang/System.<clinit> (System.java:55)
> > > >> >>>         at java/lang/J9VMInternals.initializeImpl (Native
> Method)
> > > >> >>>         at java/lang/J9VMInternals.initialize
> > > >> (J9VMInternals.java:185)
> > > >> >>>         at java/lang/ClassLoader.initializeClassLoaders
(
> > > >> ClassLoader.java:6
> > > >> >>>
> > > >> >> 5)
> > > >> >>
> > > >> >>>         at java/lang/Thread.initialize (Thread.java:301)
> > > >> >>>         at java/lang/Thread.<init> (Thread.java:120)
> > > >> >>> java/lang/NullPointerException
> > > >> >>>         at java/lang/Throwable.printStackTrace ( Throwable.java
> > > :231)
> > > >> >>>         at
> com/vladium/emma/rt/RTExitHook.createClassLoaderClosure
> > > >> (RTExitH
> > > >> >>>
> > > >> >> ook.java:63)
> > > >> >>
> > > >> >>>         at com/vladium/emma/rt/RT.reset ( RT.java:92)
> > > >> >>>         at com/vladium/emma/rt/RT.<clinit> (RT.java:238)
> > > >> >>>         at java/lang/J9VMInternals.initializeImpl (Native
> Method)
> > > >> >>>         at java/lang/J9VMInternals.initialize (
> J9VMInternals.java
> > > >> :185)
> > > >> >>>         at java/nio/Buffer.$VRi ( Buffer.java:4294967295)
> > > >> >>>         at java/nio/Buffer.<clinit> (Buffer.java:4294967295)
> > > >> >>>         at java/lang/J9VMInternals.initializeImpl (Native
> Method)
> > > >> >>>         at java/lang/J9VMInternals.initialize
> > > >> (J9VMInternals.java:185)
> > > >> >>>         at java/lang/J9VMInternals.initialize
> > > >> (J9VMInternals.java:154)
> > > >> >>>         at java/lang/String.getBytes ( String.java :783)
> > > >> >>>         at org/apache/harmony/luni/util/Util.getBytes
(
> Util.java
> > > :54)
> > > >> >>>         at java/lang/ClassLoader.loadLibraryWithPath
> > > >> (ClassLoader.java
> > > >> :998)
> > > >> >>>         at java/lang/ClassLoader.loadLibraryWithClassLoader
(
> > > >> ClassLoader.ja
> > > >> >>>
> > > >> >> va:970)
> > > >> >>
> > > >> >>>         at java/lang/System.loadLibrary ( System.java:706)
> > > >> >>>         at com/ibm/icu4jni/common/ICU4JNILoader.loadLibrary
> > > (Unknown
> > > >> Source
> > > >> >>>
> > > >> >> )
> > > >> >>
> > > >> >>>         at com/ibm/icu4jni/converters/NativeConverter.<clinit>
> > > >> (Unknown Sou
> > > >> >>>
> > > >> >> rce)
> > > >> >>
> > > >> >>>         at java/lang/J9VMInternals.initializeImpl (Native
> Method)
> > > >> >>>         at java/lang/J9VMInternals.initialize
> > > >> (J9VMInternals.java:185)
> > > >> >>>         at
> > > com/ibm/icu4jni/charset/CharsetProviderICU.charsetForName
> > > >> (Unkno
> > > >> >>>
> > > >> >> wn Source)
> > > >> >>
> > > >> >>>         at java/nio/charset/Charset.forNameInternal
> > > >> (Charset.java:450)
> > > >> >>>         at java/nio/charset/Charset.forName (Charset.java:513)
> > > >> >>>         at java/lang/String.defaultCharset ( String.java:688)
> > > >> >>>         at java/lang/String.<init> (String.java:252)
> > > >> >>>         at org/apache/harmony/luni/util/Util.toString
(
> Util.java
> > > :74)
> > > >> >>>         at java/lang/System.getPropertyList (Native Method)
> > > >> >>>         at java/lang/System.ensureProperties (System.java:496)
> > > >> >>>         at java/lang/System.<clinit> (System.java:55)
> > > >> >>>         at java/lang/J9VMInternals.initializeImpl (Native
> Method)
> > > >> >>>         at java/lang/J9VMInternals.initialize
> > > >> (J9VMInternals.java:185)
> > > >> >>>         at java/lang/ClassLoader.initializeClassLoaders
(
> > > >> ClassLoader.java:6
> > > >> >>>
> > > >> >> 5)
> > > >> >>
> > > >> >>>         at java/lang/Thread.initialize (Thread.java:301)
> > > >> >>>         at java/lang/Thread.<init> (Thread.java
:120)
> > > >> >>> JVMJ9VM015W Initialization error for library jclclear_23(14):
> > > >> JVMJ9VM009E J
> > > >> >>>
> > > >> >> 9VMDllMain failed
> > > >> >>
> > > >> >>> HMYEXEL062E Internal VM error: Failed to create Java
VM
> > > >> >>> FAILED.
> > > >> >>>
> > > >> >>>
> > > >> >>> On 1 June 2006 at 22:26, "Mikhail Loenko" < mloenko@gmail.com
>
> > > >> wrote:
> > > >> >>>
> > > >> >>>
> > > >> >>>> 2006/6/1, Geir Magnusson Jr < geir@pobox.com>:
> > > >> >>>>
> > > >> >>>>
> > > >> >>>>> Mikhail Loenko wrote:
> > > >> >>>>>
> > > >> >>>>>
> > > >> >>>>>> 2006/6/1, Geir Magnusson Jr < geir@pobox.com
>:
> > > >> >>>>>>
> > > >> >>>>>>
> > > >> >>>>>>> I don't quite grok what you mean - could
you post what you
> are
> > > >> >>>>>>> suggesting in code here on -dev?
> > > >> >>>>>>>
> > > >> >>>>>>>
> > > >> >>>>>> Index:
> > > >> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.ja
> > > >> >>>>>>
> > > >> >> v
> > > >> >>
> > > >> >>>>>>
> > > >> >>>>>>
> > > >> >>>> a
> > > >> >>>>
> > > >> >>>>
> > > >> >>>>>>
> > > >> ===================================================================
> > > >> >>>>>> ---
> > > >> >>>>>>
> > > >> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.java
> > > >> >>>>>> (revision
> > > >> >>>>>> 410480)
> > > >> >>>>>> +++
> > > >> >>>>>>
> > > >> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.java
> > > >>
> > > >> >>>>>> (working
> > > >> >>>>>> copy)
> > > >> >>>>>> @@ -785,7 +785,7 @@
> > > >> >>>>>>                 "FFFF:FFFF:FFFF:FFFF:FFFF:FFFF:
> 255.255.255.255
> > > ",
> > > >> >>>>>>                 "0:0:0:0:0:0:0:0", "0:0:0:0:0:0:
0.0.0.0" };
> > > >> >>>>>>
> > > >> >>>>>> -        String invalidIPAddresses[] = {
":", "FFFF:FFFF",
> > > "1:1",
> > > >> >>>>>> "::1.2.3.444" };
> > > >> >>>>>> +        String invalidIPAddresses[] = {
"FFFF:FFFF" };
> > > >> >>>>>>
> > > >> >>>>>>         for (int i = 0; i < validIPAddresses.length;
i++) {
> > > >> >>>>>>
> > > >> >>>>>>
> > > >> >>>>>>
> > > >> >>>>>>
> > > >> >>>>> Thx - why do you want to remove testing the others?
 Because
> you
> > > >> believe
> > > >> >>>>> that it is in effect testing the resolver in
the OS since
> each
> > > >> returns
> > > >> >>>>> the same answer,and thus it's testing the same
thing over and
> > > >> over?
> > > >> >>>>>
> > > >> >>>>>
> > > >> >>>> I believe so, but I'd like to hear confirmation from
code
> authors
> > > >> >>>>
> > > >> >>>> Thanks,
> > > >> >>>> Mikhail
> > > >> >>>>
> > > >> >>>>
> > > >> >>>
> > > >> >>>
> > > >>
> ---------------------------------------------------------------------
> > > >> >>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > >> >>> To unsubscribe, e-mail:
> harmony-dev-unsubscribe@incubator.apache.org
> > >
> > > >> >>> For additional commands, e-mail:
> > > >> harmony-dev-help@incubator.apache.org
> > > >> >>>
> > > >> >>>
> > > >> >>>
> > > >> >>>
> > > >> >> --
> > > >> >> Paulex Yang
> > > >> >> China Software Development Lab
> > > >> >> IBM
> > > >> >>
> > > >> >>
> > > >> >>
> > > >> >>
> > > ---------------------------------------------------------------------
> > > >> >> Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > >> >> To unsubscribe, e-mail:
> harmony-dev-unsubscribe@incubator.apache.org
> > > >> >> For additional commands, e-mail:
> > > >> harmony-dev-help@incubator.apache.org
> > > >> >>
> > > >> >
> > > >> >
> > > >> >
> > > >> >
> ---------------------------------------------------------------------
> > >
> > > >> > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > >> > To unsubscribe, e-mail:
> harmony-dev-unsubscribe@incubator.apache.org
> > > >> > For additional commands, e-mail:
> > > harmony-dev-help@incubator.apache.org
> > > >> >
> > > >> >
> > > >> >
> > > >>
> > > >>
> > > >> --
> > > >> Paulex Yang
> > > >> China Software Development Lab
> > > >> IBM
> > > >>
> > > >>
> > > >>
> > > >>
> ---------------------------------------------------------------------
> > > >> Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > >> To unsubscribe, e-mail:
> harmony-dev-unsubscribe@incubator.apache.org
> > > >> For additional commands, e-mail:
> harmony-dev-help@incubator.apache.org
> > > >>
> > > >>
> > > >
> > >
> > >
> > > --
> > > Paulex Yang
> > > China Software Development Lab
> > > IBM
> > >
> > >
> > >
> > > ---------------------------------------------------------------------
> > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > >
> > >
> >
> >
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message