harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mikhail Loenko" <mloe...@gmail.com>
Subject Re: classlib test suite status emails?
Date Wed, 12 Apr 2006 11:04:53 GMT
Top 12 winners out of 758 tests (1.6%) according to junit report
take 458 sec out of 572 (80%)

The winners are:

IdentityScopeTest       112.031
IdentityTest            85.233
Inet6AddressTest        57.944
NTLoginModuleTest       55.199
OutOfMemoryErrorTest    30.724
SignatureTest           26.859
ObjectTest              19.148
ThreadGroupTest         18.878
DatagramChannelTest     14.852
SocketChannelTest       14.171
TimerTest               12.408
KeyPairGenerator3Test   10.405

Thanks,
Mikhail

2006/4/12, Mikhail Loenko <mloenko@gmail.com>:
> I've fixed tests that change security manager and set of providers,
> now the fork mode for the security tests is "once", run time reduced
> by ~7 minutes.
>
> So now there are other champions, should we go through them and
> reduce/exclude/move to another suite?
>
> Thanks,
> Mikhail
>
> 2006/4/11, Mark Hindess <mark.hindess@googlemail.com>:
> > Personally, obviously,  I'd expect people to run the tests before committing.
> >
> > However, I notice that since enabling the security tests - which fork
> > for every test - that the tests take over half an hour to run now on
> > our Linux build machine.  So I can see why enthusiasm might lead to
> > people not running all the tests but instead perhaps just running
> > those from the module you are changing.
> >
> > George just commented in a commit message backing out some changes in
> > text that all the text tests passed even though the changed caused
> > failures elsewhere.
> >
> > Regards,
> >  Mark.
> >
> > On 4/11/06, Geir Magnusson Jr <geir@pobox.com> wrote:
> > > Just curious (and this isn't a criticism - I'm just as guilty of not
> > > doing this)...
> > >
> > > Don't you run the tests before committing?
> > >
> > > geir
> > >
> > > George Harley wrote:
> > > > Hi,
> > > >
> > > > It *seems* like things started failing after I committed the changes for
> > > > HARMONY-205 last night. I'm looking into this now. If the investigation
> > > > begins to take up too much time I will back the changes out.
> > > >
> > > > Best regards,
> > > > George
> > > >
> > > >
> > > > Stepan Mishura wrote:
> > > >> The same for me + DatagramChannelTest
> > > >>
> > > >> Thanks,
> > > >> Stepan.
> > > >>
> > > >> On 4/11/06, Mark Hindess wrote:
> > > >>
> > > >>> No.  These:
> > > >>>
> > > >>> F
> > > >>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoder
> > > >>>
> > > >>> E
> > > >>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder01
> > > >>>
> > > >>> E
> > > >>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder02
> > > >>>
> > > >>> F org.apache.harmony.security.asn1.der.DerUTCTimeEDTest.testMt
> > > >>> E
> > > >>> org.apache.harmony.security.x509.PrivateKeyUsagePeriodTest.testEncodeDecode
> > > >>>
> > > >>> F java.security.cert.X509CertSelectorTest.testSetPrivateKeyValid
> > > >>> F java.security.cert.X509CertSelectorTest.testMatch
> > > >>> F java.security.cert.X509CertSelectorTest.testClone
> > > >>> F tests.api.java.sql.DateTest.testSetTimelong
> > > >>> F tests.api.java.sql.DateTest.testToString
> > > >>> F tests.api.java.sql.DateTest.testValueOf
> > > >>> F tests.api.java.sql.TimestampTest.testSetNanosint
> > > >>> F tests.api.java.sql.TimestampTest.testToString
> > > >>> F tests.api.java.util.DateTest.test_toGMTString
> > > >>> F tests.api.java.util.DateTest.test_toString
> > > >>>
> > > >>> Regards,
> > > >>> Mark.
> > > >>>
> > > >>> On 4/11/06, Stepan Mishura <stepan.mishura@gmail.com> wrote:
> > > >>>
> > > >>>> On 4/11/06, Mark Hindess wrote:
> > > >>>>
> > > >>>>> Yes.  I was using the failureproperty mechanism.  Trying
to get this
> > > >>>>> property propogated back to the top level ant file was
what I was
> > > >>>>> having trouble with.
> > > >>>>>
> > > >>>>> Using a file as you suggest might help.  I'll give that
a try
> > > >>>>>
> > > >>> shortly...
> > > >>>
> > > >>>>> Incidentally, I'm seeing 12 failures and 3 errors on r393111.
> > > >>>>>
> > > >>>>  I guess that you have 9 tests from DatagramChannelTest passed.
And
> > > >>>> 12 +
> > > >>>>
> > > >>> 3 =
> > > >>>
> > > >>>> 15 :-)
> > > >>>>
> > > >>>>  (And
> > > >>>>
> > > >>>>> there are typos - "mathc" should be "match" - in the failure
messages
> > > >>>>> for java.security.cert testMatch and testClone.)
> > > >>>>>
> > > >>>> I've fixed typo.
> > > >>>>
> > > >>>> Thanks,
> > > >>>> Stepan.
> > > >>>>
> > > >>>> Regards,
> > > >>>>
> > > >>>>> Mark.
> > > >>>>>
> > > >>>>> On 4/11/06, Stepan Mishura <stepan.mishura@gmail.com>
wrote:
> > > >>>>>
> > > >>>>>> On 4/11/06, Mark Hindess wrote:
> > > >>>>>>
> > > >>>>>>
> > > >>>>>>> Stepan,
> > > >>>>>>>
> > > >>>>>>> I have another build running (but without notifications
going to
> > > >>>>>>>
> > > >>> the
> > > >>>
> > > >>>>>>> list) that runs:
> > > >>>>>>>
> > > >>>>>>> 1) build (with reference jdk)
> > > >>>>>>> 2) build with what we created with 1)
> > > >>>>>>> 3) test
> > > >>>>>>> 4) create classlists and compare with class load
data for
> > > >>>>>>>
> > > >>> applications
> > > >>>
> > > >>>>>>>    (tomcat, geronimo, continuum, etc.)
> > > >>>>>>> 5) generate JAPI results
> > > >>>>>>>
> > > >>>>>>> I'd like to fail this build at step 3, but I can't
see an easy way
> > > >>>>>>>
> > > >>> to
> > > >>>
> > > >>>>>>> get 'ant -f make/build.xml test' to run all tests
and then fail if
> > > >>>>>>>
> > > >>> any
> > > >>>
> > > >>>>>>> of the module test sub-targets had test failures.
 I could parse
> > > >>>>>>>
> > > >>> the
> > > >>>
> > > >>>>>>> output I suppose, but I'd rather get ant to propagate
the junit
> > > >>>>>>>
> > > >>> tasks
> > > >>>
> > > >>>>>>> failure property back up to the top level.  I've
tried a couple of
> > > >>>>>>> things and none seem to work.  Any suggestions
welcome.
> > > >>>>>>>
> > > >>>>>>  Mark, did you try failureproperty parameter for junit
task?
> > > >>>>>> We may add it to ant sub-targets to raise a flag,
for example,
> > > >>>>>>
> > > >>> create
> > > >>>
> > > >>>>> file "
> > > >>>>>
> > > >>>>>> TESTS.FAILED" in the root,  when tests for some module
fail. And in
> > > >>>>>>
> > > >>> the
> > > >>>
> > > >>>>> end
> > > >>>>>
> > > >>>>>> of tests suite run check whether this file exists
on not.
> > > >>>>>>
> > > >>>>>> Thanks,
> > > >>>>>> Stepan.
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> Regards,
> > > >>>>>>
> > > >>>>>>> Mark.
> > > >>>>>>>
> > > >>>>>>> On 4/11/06, Stepan Mishura <stepan.mishura@gmail.com>
wrote:
> > > >>>>>>>
> > > >>>>>>>> Hi,
> > > >>>>>>>>
> > > >>>>>>>> I've checked out at morning last updates,
built the code base
> > > >>>>>>>>
> > > >>> and
> > > >>>
> > > >>>>> run
> > > >>>>>
> > > >>>>>>> the
> > > >>>>>>>
> > > >>>>>>>> tests …and there are 24 tests failures!
> > > >>>>>>>>
> > > >>>>>>>> There are 9 tests failures in
> > > >>>>>>>> org.apache.harmony.tests.java.nio.channels.DatagramChannelTest
–
> > > >>>>>>>>
> > > >>> I
> > > >>>
> > > >>>>> saw
> > > >>>>>
> > > >>>>>>> these
> > > >>>>>>>
> > > >>>>>>>> failures before from time to time. It seems
that tests depend on
> > > >>>>>>>>
> > > >>>>> some
> > > >>>>>
> > > >>>>>>> race
> > > >>>>>>>
> > > >>>>>>>> conditions because they may pass if I rerun
them. Paulex, are
> > > >>>>>>>>
> > > >>> these
> > > >>>
> > > >>>>>>> tests
> > > >>>>>>>
> > > >>>>>>>> passing for you?
> > > >>>>>>>>
> > > >>>>>>>> And there are new 15 test failures.  So now
if I'll make a code
> > > >>>>>>>>
> > > >>>>> update
> > > >>>>>
> > > >>>>>>> or a
> > > >>>>>>>
> > > >>>>>>>> bug fix how I can be 100% sure that I don't
do any regression?
> > > >>>>>>>>
> > > >>>>>>>> Currently if a commit breaks the Harmony classlib
build a
> > > >>>>>>>>
> > > >>>>> notification
> > > >>>>>
> > > >>>>>>> with
> > > >>>>>>>
> > > >>>>>>>> subject: "[continuum] BUILD FAILURE: Classlib/linux.ia32"
will
> > > >>>>>>>>
> > > >>> be
> > > >>>
> > > >>>>> send
> > > >>>>>
> > > >>>>>>> to
> > > >>>>>>>
> > > >>>>>>>> harmony-commits mailing list. Is it possible
to have the same
> > > >>>>>>>>
> > > >>> for
> > > >>>
> > > >>>>> tests?
> > > >>>>>
> > > >>>>>>> I
> > > >>>>>>>
> > > >>>>>>>> mean that after completing automatic build
the existing classlib
> > > >>>>>>>>
> > > >>>>> tests
> > > >>>>>
> > > >>>>>>> suite
> > > >>>>>>>
> > > >>>>>>>> should be run. If there are failing tests
then a report
> > > >>>>>>>>
> > > >>> notification
> > > >>>
> > > >>>>>>> with
> > > >>>>>>>
> > > >>>>>>>> corresponding subject will be send.
> > > >>>>>>>>
> > > >>>>>>>> Thanks,
> > > >>>>>>>> Stepan Mishura
> > > >>>>>>>> Intel Middleware Products Division
> > > >>>>>>>>
> > > >>>>>>> --
> > > >>>>>>> Mark Hindess <mark.hindess@googlemail.com>
> > > >>>>>>> IBM Java Technology Centre, UK.
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>>
> > > >>> ---------------------------------------------------------------------
> > > >>>
> > > >>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > >>>>>>> To unsubscribe, e-mail:
> > > >>>>>>>
> > > >>> harmony-dev-unsubscribe@incubator.apache.org
> > > >>>
> > > >>>>>>> For additional commands, e-mail:
> > > >>>>>>>
> > > >>> harmony-dev-help@incubator.apache.org
> > > >>>
> > > >>>>>>>
> > > >>>>>> --
> > > >>>>>>
> > > >>>>>>
> > > >>> ---------------------------------------------------------------------
> > > >>>
> > > >>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > >>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > >>>>>> For additional commands, e-mail:
> > > >>>>>>
> > > >>> harmony-dev-help@incubator.apache.org
> > > >>>
> > > >>>>>> Thanks,
> > > >>>>>> Stepan Mishura
> > > >>>>>> Intel Middleware Products Division
> > > >>>>>>
> > > >>>>>>
> > > >>>>>>
> > > >>>>> --
> > > >>>>> Mark Hindess <mark.hindess@googlemail.com>
> > > >>>>> IBM Java Technology Centre, UK.
> > > >>>>>
> > > >>>>> ---------------------------------------------------------------------
> > > >>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > >>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > >>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > > >>>>>
> > > >>>>>
> > > >>>>>
> > > >>>> --
> > > >>>> ---------------------------------------------------------------------
> > > >>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > >>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > >>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > > >>>>
> > > >>>> Thanks,
> > > >>>> Stepan Mishura
> > > >>>> Intel Middleware Products Division
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>> --
> > > >>> Mark Hindess <mark.hindess@googlemail.com>
> > > >>> IBM Java Technology Centre, UK.
> > > >>>
> > > >>> ---------------------------------------------------------------------
> > > >>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > >>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > >>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > > >>>
> > > >>>
> > > >>>
> > > >>
> > > >>
> > > >> --
> > > >> ---------------------------------------------------------------------
> > > >> Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > >> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > >> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > > >>
> > > >> Thanks,
> > > >> Stepan Mishura
> > > >> Intel Middleware Products Division
> > > >>
> > > >>
> > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > > >
> > > >
> > >
> > > ---------------------------------------------------------------------
> > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > >
> > >
> >
> >
> > --
> > Mark Hindess <mark.hindess@googlemail.com>
> > IBM Java Technology Centre, UK.
> >
> > ---------------------------------------------------------------------
> > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >
> >
>

---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org


Mime
View raw message