harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mikhail Loenko" <mloe...@gmail.com>
Subject Re: [classlib] [testing] Coverage (was Re: 37% of total test execution time is spent in a single test)
Date Tue, 13 Jun 2006 11:12:01 GMT
Cool! Sorry, I've missed it

One more thing I've missed: what is the exclude list you've finally ended with?

Thanks,
Mikhail

2006/6/13, Vladimir Ivanov <ivavladimir@gmail.com>:
> The detailed info available at  http://viv.byethost15.com/ (this address
> also pointed on wiki-page).
> Do you need additional information ?
>
> Thanks,
>  Vladimir
>
>
> On 6/13/06, Mikhail Loenko <mloenko@gmail.com> wrote:
> >
> > Great!
> >
> > Is it possible to see uncovered classes/methods/lines?
> >
> > Thanks,
> > Mikhail
> >
> > P.S. I think java.awt does not currently belong to beans module,
> > though who knows what the future may bring us :)
> >
> > 2006/6/13, Vladimir Ivanov <ivavladimir@gmail.com>:
> > > Latest Harmony API source coverage by Harmony API unit tests results I
> > > stored at wiki page
> > >
> > > http://wiki.apache.org/harmony/Coverage_information
> > >
> > >
> > >
> > > I'm going to refresh it bi-weekly (seems, it is enough for coverage).
> > >
> > >
> > >
> > > >I think we have got a agreement on the test naming convention[1], but
> > > >for sure, there have been many (legacy) test cases before this
> > > >agreement, and I think the volunteer is highly welcome to provide patch
> > > >for them.
> > > >
> > >
> > >[1]http://incubator.apache.org/harmony/subcomponents/classlibrary/testing.html
> > >
> > > >
> > >
> > > If nobody objects I'm going to look through the unit tests to correct
> > > package names according to the agreement (where needed).
> > >
> > >
> > >
> > >  Thanks,
> > >
> > >   Vladimir
> > >
> > >
> > >
> > > On 6/8/06, Paulex Yang <paulex.yang@gmail.com > wrote:
> > > >
> > > > Vladimir,
> > > >
> > > > Vladimir Ivanov wrote:
> > > > > Thanks Paulex!
> > > > >
> > > > > I did the same, but could not send results due to spam filter J
> > > > > Observations:
> > > > >
> > > > >   1. Coverage results look pretty much similar.
> > > > >   2. Exclude list looks pretty much similar too, but, looks like
it
> > > > >   depends on the way of data collection (I didn't run ant task and
> > the
> > > > > list is
> > > > >   a little bit different).
> > > > Great.
> > > > >
> > > > > In any case, I think, when we run harmony on another VM exclude list
> > > > will
> > > > > have to be updated.
> > > > >
> > > > >
> > > > >
> > > > > May be we can start publishing the coverage information on wiki
> > pages
> > > > and
> > > > > provide some updates time to time (I can do it)?
> > > > >
> > > > +1,  and of course, you can only if  no one in the mailing list
> > objects
> > > > and you'll have my welcome, and I think it will be even greater if
> > these
> > > > reports can be generated regularly like what JAPI is doing:)
> > > > >
> > > > >
> > > > > One note:
> > > > >
> > > > > I noticed that different unit tests have very different package
> > names
> > > > >
> > > > > Now the directory with all built tests copied to one place looks
> > like:
> > > > I think we have got a agreement on the test naming convention[1], but
> > > > for sure, there have been many (legacy) test cases before this
> > > > agreement, and I think the volunteer is highly welcome to provide
> > patch
> > > > for them.
> > > >
> > > >
> > [1]http://incubator.apache.org/harmony/subcomponents/classlibrary/testing.html
> > > >
> > > > >
> > > > >
> > > > >
> > > > > C:\coverage\tests\test>ls
> > > > >
> > > > > GZIPOutClose2.txt        api           config         javax
> > > > > tests
> > > > >
> > > > > GZIPOutFinish.txt        api.injected  dazzle
> > org            xml
> > > >
> > > > >
> > > > > GZIPOutWrite.txt         binary        gif            prefs
> > > > >
> > > > > Inet6Address.golden.ser  bundles       impl           serialization
> > > > >
> > > > > JDK2-3gabba.zip          com           impl.injected   test.txt
> > > > >
> > > > >
> > > > >
> > > > > I think, it would be good if tests had unified package names.
> > > > >
> > > > > Why? – so far, just common sense, just to have an order in test
> > suite
> > > > >
> > > > > Organization (if consider all unit tests as solid test suite).
> > > > >
> > > > > Thanks,
> > > > >  Vladimir
> > > > >
> > > > > For example, my exclude list for java.io is:
> > > > > -java.io.BufferedInputStream ,
> > > > > -java.io.BufferedOutputStream,
> > > > > -java.io.File ,
> > > > > -java.io.FileChannelFactory,
> > > > > -java.io.FileDescriptor,
> > > > > -java.io.FileInputStream,
> > > > > -java.io.FileOutputStream,
> > > > > -java.io.FilterInputStream ,
> > > > > -java.io.FilterOutputStream,
> > > > > - java.io.InputStream,
> > > > > -java.io.OutputStream,
> > > > > -java.io.ObjectStreamField,
> > > > > -java.io.PrintStream
> > > > >
> > > > >
> > > > > On 6/6/06, Paulex Yang < paulex.yang@gmail.com > wrote:
> > > > >>
> > > > >> I've attach the scripts and excluded class lists to JIRA,  please
> > refer
> > > > >> to https://issues.apache.org/jira/browse/HARMONY-564 .  Enjoy
it:).
> > > > >>
> > > > >> Mark Hindess wrote:
> > > > >> > On 2 June 2006 at 10:37, Paulex Yang < paulex.yang@gmail.com>
> > wrote:
> > > > >> >
> > > > >> >> Mark,
> > > > >> >>
> > > > >> >> I'm glad that there is someone else has interest on
emma, I've
> > > > >> tried it
> > > > >> >> before. AFAIK, emma works by instrumentation, but sometimes
for
> > > > >> classes
> > > > >> >> in bootclasspath, the instrumentation cannot work, there
are two
> > > > >> cases:
> > > > >>
> > > > >> >> 1. Some instrumented classes cannot be loaded by VM.
> > > > >> >> 2. Some classes cannot be instrumented
> > > > >> >>
> > > > >> >> I have tried to look more inside to find some way to
work around
> > > > >> but I
> > > > >> >> haven't got enough time yet.
> > > > >> >>
> > > > >> >> Specifically for nio-channel module, I had a list for
these two
> > > > cases
> > > > >> (I
> > > > >> >> believe the data is a little outdated and should be
reevaluated)
> > > > >> >> case 1.
> > > > >> >> BaseByteBuffer.class
> > > > >> >> Buffer.class
> > > > >> >> BufferFactory.class
> > > > >> >> ByteBuffer.class
> > > > >> >> CharArrayBuffer.class
> > > > >> >> CharBuffer.class
> > > > >> >> HeapByteBuffer.class
> > > > >> >> ReadWriteCharArrayBuffer.class
> > > > >> >> ReadWriteHeapByteBuffer.class
> > > > >> >> FileChannel.class
> > > > >> >> AbstractInterruptibleChannel.class
> > > > >> >> FileChannelImpl.class
> > > > >> >> WriteOnlyFileChannel.class
> > > > >> >> LockManager.class
> > > > >> >> LockManager$1.class
> > > > >> >> ReadOnlyFileChannel.class
> > > > >> >>
> > > > >> >> case 2:
> > > > >> >> ByteChannel.class
> > > > >> >> Channel.class
> > > > >> >> GatheringByteChannel.class
> > > > >> >> InterruptibleChannel.class
> > > > >> >> WritableByteChannel.class
> > > > >> >>
> > > > >> >> And I have got some ant script and more excluded list
for emma,
> > if
> > > > >> >> anyone has interests, I can upload it to JIRA.
> > > > >> >>
> > > > >> >
> > > > >> > Yes!
> > > > >> >
> > > > >> > -Mark.
> > > > >> >
> > > > >> >
> > > > >> >> Mark Hindess wrote:
> > > > >> >>
> > > > >> >>> Anyone tried using emma (emma.sf.net) to look at
test coverage
> > > > >> for our
> > > > >> >>> tests?  That might help ensure we don't trim them
down too
> > much?
> > > > >> >>>
> > > > >> >>> I suspect we will soon reach a point where there
are
> > significantly
> > > > >> fewer
> > > > >> >>> obvious opportunities for small contributions -
e.g.
> > > > >> 1.5generification,
> > > > >> >>> etc from looking at JAPI reports.  Having test coverage
reports
> > > > >> might
> > > > >> >>> help by highlighting areas for improvement.
> > > > >> >>>
> > > > >> >>> I had a quick try with:
> > > > >> >>>
> > > > >> >>>   ant -f make/build.xml
> > > > >> >>>   ant -f modules/nio/make/build.xml test
> > > > >> >>>   java -cp emma.jar emma instr -m overwrite -cp
> > > > >> deploy/jdk/jre/lib/boot/nio
> > > > >> >>>
> > > > >> >> .jar
> > > > >> >>
> > > > >> >>> and then:
> > > > >> >>>
> > > > >> >>>   java -Xbootclasspath/p: emma.jar \
> > > > >> >>>        -cp build/tests:modules/nio/bin/test:junit.jar:.
\
> > > > >> >>>        junit.textui.TestRunner \
> > > > >> >>>
> > org.apache.harmony.tests.java.nio.channels.SocketChannelTest
> > > > >> >>>
> > > > >> >>> (Running all the tests is a little tricky because
test depends
> > on
> > > > >> build,
> > > > >> >>> which depends on clean which wipes out the instr'd
jar. ;-( )
> > > > >> >>>
> > > > >> >>> Sadly it dies horribly with the stack trace below.
 I'll take
> > > > >> another
> > > > >> >>> look when I have a little more time, but I thought
I'd mention
> > it
> > > > in
> > > > >> >>> case anyone else had any thoughts on the subject.
> > > > >> >>>
> > > > >> >>> Regards,
> > > > >> >>>
> > > > >> >>> Mark.
> > > > >> >>>
> > > > >> >>> Exception in thread "main"
> > java/lang/ExceptionInInitializerError
> > > > >> >>>         at java/lang/J9VMInternals.initialize
> > > > >> (J9VMInternals.java:203)
> > > > >> >>>         at java/nio/Buffer.$VRi ( Buffer.java:4294967295)
> > > > >> >>>         at java/nio/Buffer.<clinit> ( Buffer.java:4294967295)
> > > > >> >>>         at java/lang/J9VMInternals.initializeImpl
(Native
> > Method)
> > > > >> >>>         at java/lang/J9VMInternals.initialize (
> > J9VMInternals.java
> > > > >> :185)
> > > > >> >>>         at java/lang/J9VMInternals.initialize
> > > > >> (J9VMInternals.java:154)
> > > > >> >>>         at java/lang/String.getBytes (String.java:783)
> > > > >> >>>         at org/apache/harmony/luni/util/Util.getBytes
(
> > > > >> Util.java:54)
> > > > >> >>>         at java/lang/ClassLoader.loadLibraryWithPath
> > > > >> (ClassLoader.java
> > > > >> :998)
> > > > >> >>>         at java/lang/ClassLoader.loadLibraryWithClassLoader
(
> > > > >> ClassLoader.ja
> > > > >> >>>
> > > > >> >> va:970)
> > > > >> >>
> > > > >> >>>         at java/lang/System.loadLibrary (System.java:706)
> > > > >> >>>         at com/ibm/icu4jni/common/ICU4JNILoader.loadLibrary
> > > > (Unknown
> > > > >> Source
> > > > >> >>>
> > > > >> >> )
> > > > >> >>
> > > > >> >>>         at com/ibm/icu4jni/converters/NativeConverter.<clinit>
> > > > >> (Unknown Sou
> > > > >> >>>
> > > > >> >> rce)
> > > > >> >>
> > > > >> >>>         at java/lang/J9VMInternals.initializeImpl
(Native
> > Method)
> > > > >> >>>         at java/lang/J9VMInternals.initialize
> > > > >> ( J9VMInternals.java:185)
> > > > >> >>>         at
> > > > com/ibm/icu4jni/charset/CharsetProviderICU.charsetForName
> > > > >> (Unkno
> > > > >> >>>
> > > > >> >> wn Source)
> > > > >> >>
> > > > >> >>>         at java/nio/charset/Charset.forNameInternal
> > > > >> (Charset.java:450)
> > > > >> >>>         at java/nio/charset/Charset.forName (Charset.java:513)
> > > > >> >>>         at java/lang/String.defaultCharset ( String.java:688)
> > > > >> >>>         at java/lang/String.<init> ( String.java:252)
> > > > >> >>>         at org/apache/harmony/luni/util/Util.toString
(
> > Util.java
> > > > :74)
> > > > >> >>>         at java/lang/System.getPropertyList (Native
Method)
> > > > >> >>>         at java/lang/System.ensureProperties ( System.java:496)
> > > > >> >>>         at java/lang/System.<clinit> (System.java:55)
> > > > >> >>>         at java/lang/J9VMInternals.initializeImpl
(Native
> > Method)
> > > > >> >>>         at java/lang/J9VMInternals.initialize
> > > > >> (J9VMInternals.java:185)
> > > > >> >>>         at java/lang/ClassLoader.initializeClassLoaders
(
> > > > >> ClassLoader.java:6
> > > > >> >>>
> > > > >> >> 5)
> > > > >> >>
> > > > >> >>>         at java/lang/Thread.initialize (Thread.java:301)
> > > > >> >>>         at java/lang/Thread.<init> (Thread.java:120)
> > > > >> >>> java/lang/NullPointerException
> > > > >> >>>         at java/lang/Throwable.printStackTrace (
Throwable.java
> > > > :231)
> > > > >> >>>         at
> > com/vladium/emma/rt/RTExitHook.createClassLoaderClosure
> > > > >> (RTExitH
> > > > >> >>>
> > > > >> >> ook.java:63)
> > > > >> >>
> > > > >> >>>         at com/vladium/emma/rt/RT.reset ( RT.java:92)
> > > > >> >>>         at com/vladium/emma/rt/RT.<clinit>
(RT.java:238)
> > > > >> >>>         at java/lang/J9VMInternals.initializeImpl
(Native
> > Method)
> > > > >> >>>         at java/lang/J9VMInternals.initialize (
> > J9VMInternals.java
> > > > >> :185)
> > > > >> >>>         at java/nio/Buffer.$VRi ( Buffer.java:4294967295)
> > > > >> >>>         at java/nio/Buffer.<clinit> (Buffer.java:4294967295)
> > > > >> >>>         at java/lang/J9VMInternals.initializeImpl
(Native
> > Method)
> > > > >> >>>         at java/lang/J9VMInternals.initialize
> > > > >> (J9VMInternals.java:185)
> > > > >> >>>         at java/lang/J9VMInternals.initialize
> > > > >> (J9VMInternals.java:154)
> > > > >> >>>         at java/lang/String.getBytes ( String.java
:783)
> > > > >> >>>         at org/apache/harmony/luni/util/Util.getBytes
(
> > Util.java
> > > > :54)
> > > > >> >>>         at java/lang/ClassLoader.loadLibraryWithPath
> > > > >> (ClassLoader.java
> > > > >> :998)
> > > > >> >>>         at java/lang/ClassLoader.loadLibraryWithClassLoader
(
> > > > >> ClassLoader.ja
> > > > >> >>>
> > > > >> >> va:970)
> > > > >> >>
> > > > >> >>>         at java/lang/System.loadLibrary ( System.java:706)
> > > > >> >>>         at com/ibm/icu4jni/common/ICU4JNILoader.loadLibrary
> > > > (Unknown
> > > > >> Source
> > > > >> >>>
> > > > >> >> )
> > > > >> >>
> > > > >> >>>         at com/ibm/icu4jni/converters/NativeConverter.<clinit>
> > > > >> (Unknown Sou
> > > > >> >>>
> > > > >> >> rce)
> > > > >> >>
> > > > >> >>>         at java/lang/J9VMInternals.initializeImpl
(Native
> > Method)
> > > > >> >>>         at java/lang/J9VMInternals.initialize
> > > > >> (J9VMInternals.java:185)
> > > > >> >>>         at
> > > > com/ibm/icu4jni/charset/CharsetProviderICU.charsetForName
> > > > >> (Unkno
> > > > >> >>>
> > > > >> >> wn Source)
> > > > >> >>
> > > > >> >>>         at java/nio/charset/Charset.forNameInternal
> > > > >> (Charset.java:450)
> > > > >> >>>         at java/nio/charset/Charset.forName (Charset.java:513)
> > > > >> >>>         at java/lang/String.defaultCharset ( String.java:688)
> > > > >> >>>         at java/lang/String.<init> (String.java:252)
> > > > >> >>>         at org/apache/harmony/luni/util/Util.toString
(
> > Util.java
> > > > :74)
> > > > >> >>>         at java/lang/System.getPropertyList (Native
Method)
> > > > >> >>>         at java/lang/System.ensureProperties (System.java:496)
> > > > >> >>>         at java/lang/System.<clinit> (System.java:55)
> > > > >> >>>         at java/lang/J9VMInternals.initializeImpl
(Native
> > Method)
> > > > >> >>>         at java/lang/J9VMInternals.initialize
> > > > >> (J9VMInternals.java:185)
> > > > >> >>>         at java/lang/ClassLoader.initializeClassLoaders
(
> > > > >> ClassLoader.java:6
> > > > >> >>>
> > > > >> >> 5)
> > > > >> >>
> > > > >> >>>         at java/lang/Thread.initialize (Thread.java:301)
> > > > >> >>>         at java/lang/Thread.<init> (Thread.java
:120)
> > > > >> >>> JVMJ9VM015W Initialization error for library jclclear_23(14):
> > > > >> JVMJ9VM009E J
> > > > >> >>>
> > > > >> >> 9VMDllMain failed
> > > > >> >>
> > > > >> >>> HMYEXEL062E Internal VM error: Failed to create
Java VM
> > > > >> >>> FAILED.
> > > > >> >>>
> > > > >> >>>
> > > > >> >>> On 1 June 2006 at 22:26, "Mikhail Loenko" < mloenko@gmail.com
>
> > > > >> wrote:
> > > > >> >>>
> > > > >> >>>
> > > > >> >>>> 2006/6/1, Geir Magnusson Jr < geir@pobox.com>:
> > > > >> >>>>
> > > > >> >>>>
> > > > >> >>>>> Mikhail Loenko wrote:
> > > > >> >>>>>
> > > > >> >>>>>
> > > > >> >>>>>> 2006/6/1, Geir Magnusson Jr < geir@pobox.com
>:
> > > > >> >>>>>>
> > > > >> >>>>>>
> > > > >> >>>>>>> I don't quite grok what you mean
- could you post what you
> > are
> > > > >> >>>>>>> suggesting in code here on -dev?
> > > > >> >>>>>>>
> > > > >> >>>>>>>
> > > > >> >>>>>> Index:
> > > > >> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.ja
> > > > >> >>>>>>
> > > > >> >> v
> > > > >> >>
> > > > >> >>>>>>
> > > > >> >>>>>>
> > > > >> >>>> a
> > > > >> >>>>
> > > > >> >>>>
> > > > >> >>>>>>
> > > > >> ===================================================================
> > > > >> >>>>>> ---
> > > > >> >>>>>>
> > > > >> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.java
> > > > >> >>>>>> (revision
> > > > >> >>>>>> 410480)
> > > > >> >>>>>> +++
> > > > >> >>>>>>
> > > > >> modules/luni/src/test/java/tests/api/java/net/Inet6AddressTest.java
> > > > >>
> > > > >> >>>>>> (working
> > > > >> >>>>>> copy)
> > > > >> >>>>>> @@ -785,7 +785,7 @@
> > > > >> >>>>>>                 "FFFF:FFFF:FFFF:FFFF:FFFF:FFFF:
> > 255.255.255.255
> > > > ",
> > > > >> >>>>>>                 "0:0:0:0:0:0:0:0", "0:0:0:0:0:0:
0.0.0.0" };
> > > > >> >>>>>>
> > > > >> >>>>>> -        String invalidIPAddresses[]
= { ":", "FFFF:FFFF",
> > > > "1:1",
> > > > >> >>>>>> "::1.2.3.444" };
> > > > >> >>>>>> +        String invalidIPAddresses[]
= { "FFFF:FFFF" };
> > > > >> >>>>>>
> > > > >> >>>>>>         for (int i = 0; i < validIPAddresses.length;
i++) {
> > > > >> >>>>>>
> > > > >> >>>>>>
> > > > >> >>>>>>
> > > > >> >>>>>>
> > > > >> >>>>> Thx - why do you want to remove testing
the others?  Because
> > you
> > > > >> believe
> > > > >> >>>>> that it is in effect testing the resolver
in the OS since
> > each
> > > > >> returns
> > > > >> >>>>> the same answer,and thus it's testing the
same thing over and
> > > > >> over?
> > > > >> >>>>>
> > > > >> >>>>>
> > > > >> >>>> I believe so, but I'd like to hear confirmation
from code
> > authors
> > > > >> >>>>
> > > > >> >>>> Thanks,
> > > > >> >>>> Mikhail
> > > > >> >>>>
> > > > >> >>>>
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > ---------------------------------------------------------------------
> > > > >> >>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > > >> >>> To unsubscribe, e-mail:
> > harmony-dev-unsubscribe@incubator.apache.org
> > > >
> > > > >> >>> For additional commands, e-mail:
> > > > >> harmony-dev-help@incubator.apache.org
> > > > >> >>>
> > > > >> >>>
> > > > >> >>>
> > > > >> >>>
> > > > >> >> --
> > > > >> >> Paulex Yang
> > > > >> >> China Software Development Lab
> > > > >> >> IBM
> > > > >> >>
> > > > >> >>
> > > > >> >>
> > > > >> >>
> > > > ---------------------------------------------------------------------
> > > > >> >> Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > > >> >> To unsubscribe, e-mail:
> > harmony-dev-unsubscribe@incubator.apache.org
> > > > >> >> For additional commands, e-mail:
> > > > >> harmony-dev-help@incubator.apache.org
> > > > >> >>
> > > > >> >
> > > > >> >
> > > > >> >
> > > > >> >
> > ---------------------------------------------------------------------
> > > >
> > > > >> > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > > >> > To unsubscribe, e-mail:
> > harmony-dev-unsubscribe@incubator.apache.org
> > > > >> > For additional commands, e-mail:
> > > > harmony-dev-help@incubator.apache.org
> > > > >> >
> > > > >> >
> > > > >> >
> > > > >>
> > > > >>
> > > > >> --
> > > > >> Paulex Yang
> > > > >> China Software Development Lab
> > > > >> IBM
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > ---------------------------------------------------------------------
> > > > >> Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > > >> To unsubscribe, e-mail:
> > harmony-dev-unsubscribe@incubator.apache.org
> > > > >> For additional commands, e-mail:
> > harmony-dev-help@incubator.apache.org
> > > > >>
> > > > >>
> > > > >
> > > >
> > > >
> > > > --
> > > > Paulex Yang
> > > > China Software Development Lab
> > > > IBM
> > > >
> > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > > >
> > > >
> > >
> > >
> >
> > ---------------------------------------------------------------------
> > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >
> >
>
>

---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org


Mime
View raw message