harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mikhail Loenko" <mloe...@gmail.com>
Subject Re: classlib test suite status emails?
Date Wed, 12 Apr 2006 10:26:26 GMT
I've fixed tests that change security manager and set of providers,
now the fork mode for the security tests is "once", run time reduced
by ~7 minutes.

So now there are other champions, should we go through them and
reduce/exclude/move to another suite?

Thanks,
Mikhail

2006/4/11, Mark Hindess <mark.hindess@googlemail.com>:
> Personally, obviously,  I'd expect people to run the tests before committing.
>
> However, I notice that since enabling the security tests - which fork
> for every test - that the tests take over half an hour to run now on
> our Linux build machine.  So I can see why enthusiasm might lead to
> people not running all the tests but instead perhaps just running
> those from the module you are changing.
>
> George just commented in a commit message backing out some changes in
> text that all the text tests passed even though the changed caused
> failures elsewhere.
>
> Regards,
>  Mark.
>
> On 4/11/06, Geir Magnusson Jr <geir@pobox.com> wrote:
> > Just curious (and this isn't a criticism - I'm just as guilty of not
> > doing this)...
> >
> > Don't you run the tests before committing?
> >
> > geir
> >
> > George Harley wrote:
> > > Hi,
> > >
> > > It *seems* like things started failing after I committed the changes for
> > > HARMONY-205 last night. I'm looking into this now. If the investigation
> > > begins to take up too much time I will back the changes out.
> > >
> > > Best regards,
> > > George
> > >
> > >
> > > Stepan Mishura wrote:
> > >> The same for me + DatagramChannelTest
> > >>
> > >> Thanks,
> > >> Stepan.
> > >>
> > >> On 4/11/06, Mark Hindess wrote:
> > >>
> > >>> No.  These:
> > >>>
> > >>> F
> > >>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoder
> > >>>
> > >>> E
> > >>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder01
> > >>>
> > >>> E
> > >>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder02
> > >>>
> > >>> F org.apache.harmony.security.asn1.der.DerUTCTimeEDTest.testMt
> > >>> E
> > >>> org.apache.harmony.security.x509.PrivateKeyUsagePeriodTest.testEncodeDecode
> > >>>
> > >>> F java.security.cert.X509CertSelectorTest.testSetPrivateKeyValid
> > >>> F java.security.cert.X509CertSelectorTest.testMatch
> > >>> F java.security.cert.X509CertSelectorTest.testClone
> > >>> F tests.api.java.sql.DateTest.testSetTimelong
> > >>> F tests.api.java.sql.DateTest.testToString
> > >>> F tests.api.java.sql.DateTest.testValueOf
> > >>> F tests.api.java.sql.TimestampTest.testSetNanosint
> > >>> F tests.api.java.sql.TimestampTest.testToString
> > >>> F tests.api.java.util.DateTest.test_toGMTString
> > >>> F tests.api.java.util.DateTest.test_toString
> > >>>
> > >>> Regards,
> > >>> Mark.
> > >>>
> > >>> On 4/11/06, Stepan Mishura <stepan.mishura@gmail.com> wrote:
> > >>>
> > >>>> On 4/11/06, Mark Hindess wrote:
> > >>>>
> > >>>>> Yes.  I was using the failureproperty mechanism.  Trying to
get this
> > >>>>> property propogated back to the top level ant file was what
I was
> > >>>>> having trouble with.
> > >>>>>
> > >>>>> Using a file as you suggest might help.  I'll give that a try
> > >>>>>
> > >>> shortly...
> > >>>
> > >>>>> Incidentally, I'm seeing 12 failures and 3 errors on r393111.
> > >>>>>
> > >>>>  I guess that you have 9 tests from DatagramChannelTest passed.
And
> > >>>> 12 +
> > >>>>
> > >>> 3 =
> > >>>
> > >>>> 15 :-)
> > >>>>
> > >>>>  (And
> > >>>>
> > >>>>> there are typos - "mathc" should be "match" - in the failure
messages
> > >>>>> for java.security.cert testMatch and testClone.)
> > >>>>>
> > >>>> I've fixed typo.
> > >>>>
> > >>>> Thanks,
> > >>>> Stepan.
> > >>>>
> > >>>> Regards,
> > >>>>
> > >>>>> Mark.
> > >>>>>
> > >>>>> On 4/11/06, Stepan Mishura <stepan.mishura@gmail.com>
wrote:
> > >>>>>
> > >>>>>> On 4/11/06, Mark Hindess wrote:
> > >>>>>>
> > >>>>>>
> > >>>>>>> Stepan,
> > >>>>>>>
> > >>>>>>> I have another build running (but without notifications
going to
> > >>>>>>>
> > >>> the
> > >>>
> > >>>>>>> list) that runs:
> > >>>>>>>
> > >>>>>>> 1) build (with reference jdk)
> > >>>>>>> 2) build with what we created with 1)
> > >>>>>>> 3) test
> > >>>>>>> 4) create classlists and compare with class load data
for
> > >>>>>>>
> > >>> applications
> > >>>
> > >>>>>>>    (tomcat, geronimo, continuum, etc.)
> > >>>>>>> 5) generate JAPI results
> > >>>>>>>
> > >>>>>>> I'd like to fail this build at step 3, but I can't
see an easy way
> > >>>>>>>
> > >>> to
> > >>>
> > >>>>>>> get 'ant -f make/build.xml test' to run all tests and
then fail if
> > >>>>>>>
> > >>> any
> > >>>
> > >>>>>>> of the module test sub-targets had test failures. 
I could parse
> > >>>>>>>
> > >>> the
> > >>>
> > >>>>>>> output I suppose, but I'd rather get ant to propagate
the junit
> > >>>>>>>
> > >>> tasks
> > >>>
> > >>>>>>> failure property back up to the top level.  I've tried
a couple of
> > >>>>>>> things and none seem to work.  Any suggestions welcome.
> > >>>>>>>
> > >>>>>>  Mark, did you try failureproperty parameter for junit
task?
> > >>>>>> We may add it to ant sub-targets to raise a flag, for example,
> > >>>>>>
> > >>> create
> > >>>
> > >>>>> file "
> > >>>>>
> > >>>>>> TESTS.FAILED" in the root,  when tests for some module
fail. And in
> > >>>>>>
> > >>> the
> > >>>
> > >>>>> end
> > >>>>>
> > >>>>>> of tests suite run check whether this file exists on not.
> > >>>>>>
> > >>>>>> Thanks,
> > >>>>>> Stepan.
> > >>>>>>
> > >>>>>>
> > >>>>>> Regards,
> > >>>>>>
> > >>>>>>> Mark.
> > >>>>>>>
> > >>>>>>> On 4/11/06, Stepan Mishura <stepan.mishura@gmail.com>
wrote:
> > >>>>>>>
> > >>>>>>>> Hi,
> > >>>>>>>>
> > >>>>>>>> I've checked out at morning last updates, built
the code base
> > >>>>>>>>
> > >>> and
> > >>>
> > >>>>> run
> > >>>>>
> > >>>>>>> the
> > >>>>>>>
> > >>>>>>>> tests …and there are 24 tests failures!
> > >>>>>>>>
> > >>>>>>>> There are 9 tests failures in
> > >>>>>>>> org.apache.harmony.tests.java.nio.channels.DatagramChannelTest
–
> > >>>>>>>>
> > >>> I
> > >>>
> > >>>>> saw
> > >>>>>
> > >>>>>>> these
> > >>>>>>>
> > >>>>>>>> failures before from time to time. It seems that
tests depend on
> > >>>>>>>>
> > >>>>> some
> > >>>>>
> > >>>>>>> race
> > >>>>>>>
> > >>>>>>>> conditions because they may pass if I rerun them.
Paulex, are
> > >>>>>>>>
> > >>> these
> > >>>
> > >>>>>>> tests
> > >>>>>>>
> > >>>>>>>> passing for you?
> > >>>>>>>>
> > >>>>>>>> And there are new 15 test failures.  So now if
I'll make a code
> > >>>>>>>>
> > >>>>> update
> > >>>>>
> > >>>>>>> or a
> > >>>>>>>
> > >>>>>>>> bug fix how I can be 100% sure that I don't do
any regression?
> > >>>>>>>>
> > >>>>>>>> Currently if a commit breaks the Harmony classlib
build a
> > >>>>>>>>
> > >>>>> notification
> > >>>>>
> > >>>>>>> with
> > >>>>>>>
> > >>>>>>>> subject: "[continuum] BUILD FAILURE: Classlib/linux.ia32"
will
> > >>>>>>>>
> > >>> be
> > >>>
> > >>>>> send
> > >>>>>
> > >>>>>>> to
> > >>>>>>>
> > >>>>>>>> harmony-commits mailing list. Is it possible to
have the same
> > >>>>>>>>
> > >>> for
> > >>>
> > >>>>> tests?
> > >>>>>
> > >>>>>>> I
> > >>>>>>>
> > >>>>>>>> mean that after completing automatic build the
existing classlib
> > >>>>>>>>
> > >>>>> tests
> > >>>>>
> > >>>>>>> suite
> > >>>>>>>
> > >>>>>>>> should be run. If there are failing tests then
a report
> > >>>>>>>>
> > >>> notification
> > >>>
> > >>>>>>> with
> > >>>>>>>
> > >>>>>>>> corresponding subject will be send.
> > >>>>>>>>
> > >>>>>>>> Thanks,
> > >>>>>>>> Stepan Mishura
> > >>>>>>>> Intel Middleware Products Division
> > >>>>>>>>
> > >>>>>>> --
> > >>>>>>> Mark Hindess <mark.hindess@googlemail.com>
> > >>>>>>> IBM Java Technology Centre, UK.
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>> ---------------------------------------------------------------------
> > >>>
> > >>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> > >>>>>>> To unsubscribe, e-mail:
> > >>>>>>>
> > >>> harmony-dev-unsubscribe@incubator.apache.org
> > >>>
> > >>>>>>> For additional commands, e-mail:
> > >>>>>>>
> > >>> harmony-dev-help@incubator.apache.org
> > >>>
> > >>>>>>>
> > >>>>>> --
> > >>>>>>
> > >>>>>>
> > >>> ---------------------------------------------------------------------
> > >>>
> > >>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> > >>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > >>>>>> For additional commands, e-mail:
> > >>>>>>
> > >>> harmony-dev-help@incubator.apache.org
> > >>>
> > >>>>>> Thanks,
> > >>>>>> Stepan Mishura
> > >>>>>> Intel Middleware Products Division
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>> --
> > >>>>> Mark Hindess <mark.hindess@googlemail.com>
> > >>>>> IBM Java Technology Centre, UK.
> > >>>>>
> > >>>>> ---------------------------------------------------------------------
> > >>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> > >>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > >>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>> --
> > >>>> ---------------------------------------------------------------------
> > >>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> > >>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > >>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > >>>>
> > >>>> Thanks,
> > >>>> Stepan Mishura
> > >>>> Intel Middleware Products Division
> > >>>>
> > >>>>
> > >>>>
> > >>> --
> > >>> Mark Hindess <mark.hindess@googlemail.com>
> > >>> IBM Java Technology Centre, UK.
> > >>>
> > >>> ---------------------------------------------------------------------
> > >>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> > >>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > >>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > >>>
> > >>>
> > >>>
> > >>
> > >>
> > >> --
> > >> ---------------------------------------------------------------------
> > >> Terms of use : http://incubator.apache.org/harmony/mailing.html
> > >> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > >> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > >>
> > >> Thanks,
> > >> Stepan Mishura
> > >> Intel Middleware Products Division
> > >>
> > >>
> > >
> > >
> > > ---------------------------------------------------------------------
> > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > >
> > >
> >
> > ---------------------------------------------------------------------
> > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >
> >
>
>
> --
> Mark Hindess <mark.hindess@googlemail.com>
> IBM Java Technology Centre, UK.
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>

---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org


Mime
View raw message