harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mark Hindess" <mark.hind...@googlemail.com>
Subject Re: classlib test suite status emails?
Date Tue, 11 Apr 2006 10:48:47 GMT
Personally, obviously,  I'd expect people to run the tests before committing.

However, I notice that since enabling the security tests - which fork
for every test - that the tests take over half an hour to run now on
our Linux build machine.  So I can see why enthusiasm might lead to
people not running all the tests but instead perhaps just running
those from the module you are changing.

George just commented in a commit message backing out some changes in
text that all the text tests passed even though the changed caused
failures elsewhere.

Regards,
 Mark.

On 4/11/06, Geir Magnusson Jr <geir@pobox.com> wrote:
> Just curious (and this isn't a criticism - I'm just as guilty of not
> doing this)...
>
> Don't you run the tests before committing?
>
> geir
>
> George Harley wrote:
> > Hi,
> >
> > It *seems* like things started failing after I committed the changes for
> > HARMONY-205 last night. I'm looking into this now. If the investigation
> > begins to take up too much time I will back the changes out.
> >
> > Best regards,
> > George
> >
> >
> > Stepan Mishura wrote:
> >> The same for me + DatagramChannelTest
> >>
> >> Thanks,
> >> Stepan.
> >>
> >> On 4/11/06, Mark Hindess wrote:
> >>
> >>> No.  These:
> >>>
> >>> F
> >>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoder
> >>>
> >>> E
> >>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder01
> >>>
> >>> E
> >>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder02
> >>>
> >>> F org.apache.harmony.security.asn1.der.DerUTCTimeEDTest.testMt
> >>> E
> >>> org.apache.harmony.security.x509.PrivateKeyUsagePeriodTest.testEncodeDecode
> >>>
> >>> F java.security.cert.X509CertSelectorTest.testSetPrivateKeyValid
> >>> F java.security.cert.X509CertSelectorTest.testMatch
> >>> F java.security.cert.X509CertSelectorTest.testClone
> >>> F tests.api.java.sql.DateTest.testSetTimelong
> >>> F tests.api.java.sql.DateTest.testToString
> >>> F tests.api.java.sql.DateTest.testValueOf
> >>> F tests.api.java.sql.TimestampTest.testSetNanosint
> >>> F tests.api.java.sql.TimestampTest.testToString
> >>> F tests.api.java.util.DateTest.test_toGMTString
> >>> F tests.api.java.util.DateTest.test_toString
> >>>
> >>> Regards,
> >>> Mark.
> >>>
> >>> On 4/11/06, Stepan Mishura <stepan.mishura@gmail.com> wrote:
> >>>
> >>>> On 4/11/06, Mark Hindess wrote:
> >>>>
> >>>>> Yes.  I was using the failureproperty mechanism.  Trying to get
this
> >>>>> property propogated back to the top level ant file was what I was
> >>>>> having trouble with.
> >>>>>
> >>>>> Using a file as you suggest might help.  I'll give that a try
> >>>>>
> >>> shortly...
> >>>
> >>>>> Incidentally, I'm seeing 12 failures and 3 errors on r393111.
> >>>>>
> >>>>  I guess that you have 9 tests from DatagramChannelTest passed. And
> >>>> 12 +
> >>>>
> >>> 3 =
> >>>
> >>>> 15 :-)
> >>>>
> >>>>  (And
> >>>>
> >>>>> there are typos - "mathc" should be "match" - in the failure messages
> >>>>> for java.security.cert testMatch and testClone.)
> >>>>>
> >>>> I've fixed typo.
> >>>>
> >>>> Thanks,
> >>>> Stepan.
> >>>>
> >>>> Regards,
> >>>>
> >>>>> Mark.
> >>>>>
> >>>>> On 4/11/06, Stepan Mishura <stepan.mishura@gmail.com> wrote:
> >>>>>
> >>>>>> On 4/11/06, Mark Hindess wrote:
> >>>>>>
> >>>>>>
> >>>>>>> Stepan,
> >>>>>>>
> >>>>>>> I have another build running (but without notifications
going to
> >>>>>>>
> >>> the
> >>>
> >>>>>>> list) that runs:
> >>>>>>>
> >>>>>>> 1) build (with reference jdk)
> >>>>>>> 2) build with what we created with 1)
> >>>>>>> 3) test
> >>>>>>> 4) create classlists and compare with class load data for
> >>>>>>>
> >>> applications
> >>>
> >>>>>>>    (tomcat, geronimo, continuum, etc.)
> >>>>>>> 5) generate JAPI results
> >>>>>>>
> >>>>>>> I'd like to fail this build at step 3, but I can't see an
easy way
> >>>>>>>
> >>> to
> >>>
> >>>>>>> get 'ant -f make/build.xml test' to run all tests and then
fail if
> >>>>>>>
> >>> any
> >>>
> >>>>>>> of the module test sub-targets had test failures.  I could
parse
> >>>>>>>
> >>> the
> >>>
> >>>>>>> output I suppose, but I'd rather get ant to propagate the
junit
> >>>>>>>
> >>> tasks
> >>>
> >>>>>>> failure property back up to the top level.  I've tried a
couple of
> >>>>>>> things and none seem to work.  Any suggestions welcome.
> >>>>>>>
> >>>>>>  Mark, did you try failureproperty parameter for junit task?
> >>>>>> We may add it to ant sub-targets to raise a flag, for example,
> >>>>>>
> >>> create
> >>>
> >>>>> file "
> >>>>>
> >>>>>> TESTS.FAILED" in the root,  when tests for some module fail.
And in
> >>>>>>
> >>> the
> >>>
> >>>>> end
> >>>>>
> >>>>>> of tests suite run check whether this file exists on not.
> >>>>>>
> >>>>>> Thanks,
> >>>>>> Stepan.
> >>>>>>
> >>>>>>
> >>>>>> Regards,
> >>>>>>
> >>>>>>> Mark.
> >>>>>>>
> >>>>>>> On 4/11/06, Stepan Mishura <stepan.mishura@gmail.com>
wrote:
> >>>>>>>
> >>>>>>>> Hi,
> >>>>>>>>
> >>>>>>>> I've checked out at morning last updates, built the
code base
> >>>>>>>>
> >>> and
> >>>
> >>>>> run
> >>>>>
> >>>>>>> the
> >>>>>>>
> >>>>>>>> tests …and there are 24 tests failures!
> >>>>>>>>
> >>>>>>>> There are 9 tests failures in
> >>>>>>>> org.apache.harmony.tests.java.nio.channels.DatagramChannelTest
–
> >>>>>>>>
> >>> I
> >>>
> >>>>> saw
> >>>>>
> >>>>>>> these
> >>>>>>>
> >>>>>>>> failures before from time to time. It seems that tests
depend on
> >>>>>>>>
> >>>>> some
> >>>>>
> >>>>>>> race
> >>>>>>>
> >>>>>>>> conditions because they may pass if I rerun them. Paulex,
are
> >>>>>>>>
> >>> these
> >>>
> >>>>>>> tests
> >>>>>>>
> >>>>>>>> passing for you?
> >>>>>>>>
> >>>>>>>> And there are new 15 test failures.  So now if I'll
make a code
> >>>>>>>>
> >>>>> update
> >>>>>
> >>>>>>> or a
> >>>>>>>
> >>>>>>>> bug fix how I can be 100% sure that I don't do any regression?
> >>>>>>>>
> >>>>>>>> Currently if a commit breaks the Harmony classlib build
a
> >>>>>>>>
> >>>>> notification
> >>>>>
> >>>>>>> with
> >>>>>>>
> >>>>>>>> subject: "[continuum] BUILD FAILURE: Classlib/linux.ia32"
will
> >>>>>>>>
> >>> be
> >>>
> >>>>> send
> >>>>>
> >>>>>>> to
> >>>>>>>
> >>>>>>>> harmony-commits mailing list. Is it possible to have
the same
> >>>>>>>>
> >>> for
> >>>
> >>>>> tests?
> >>>>>
> >>>>>>> I
> >>>>>>>
> >>>>>>>> mean that after completing automatic build the existing
classlib
> >>>>>>>>
> >>>>> tests
> >>>>>
> >>>>>>> suite
> >>>>>>>
> >>>>>>>> should be run. If there are failing tests then a report
> >>>>>>>>
> >>> notification
> >>>
> >>>>>>> with
> >>>>>>>
> >>>>>>>> corresponding subject will be send.
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> Stepan Mishura
> >>>>>>>> Intel Middleware Products Division
> >>>>>>>>
> >>>>>>> --
> >>>>>>> Mark Hindess <mark.hindess@googlemail.com>
> >>>>>>> IBM Java Technology Centre, UK.
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>> ---------------------------------------------------------------------
> >>>
> >>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >>>>>>> To unsubscribe, e-mail:
> >>>>>>>
> >>> harmony-dev-unsubscribe@incubator.apache.org
> >>>
> >>>>>>> For additional commands, e-mail:
> >>>>>>>
> >>> harmony-dev-help@incubator.apache.org
> >>>
> >>>>>>>
> >>>>>> --
> >>>>>>
> >>>>>>
> >>> ---------------------------------------------------------------------
> >>>
> >>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> >>>>>> For additional commands, e-mail:
> >>>>>>
> >>> harmony-dev-help@incubator.apache.org
> >>>
> >>>>>> Thanks,
> >>>>>> Stepan Mishura
> >>>>>> Intel Middleware Products Division
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>> --
> >>>>> Mark Hindess <mark.hindess@googlemail.com>
> >>>>> IBM Java Technology Centre, UK.
> >>>>>
> >>>>> ---------------------------------------------------------------------
> >>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> >>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >>>>>
> >>>>>
> >>>>>
> >>>> --
> >>>> ---------------------------------------------------------------------
> >>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> >>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >>>>
> >>>> Thanks,
> >>>> Stepan Mishura
> >>>> Intel Middleware Products Division
> >>>>
> >>>>
> >>>>
> >>> --
> >>> Mark Hindess <mark.hindess@googlemail.com>
> >>> IBM Java Technology Centre, UK.
> >>>
> >>> ---------------------------------------------------------------------
> >>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> >>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >>>
> >>>
> >>>
> >>
> >>
> >> --
> >> ---------------------------------------------------------------------
> >> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> >> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >>
> >> Thanks,
> >> Stepan Mishura
> >> Intel Middleware Products Division
> >>
> >>
> >
> >
> > ---------------------------------------------------------------------
> > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>


--
Mark Hindess <mark.hindess@googlemail.com>
IBM Java Technology Centre, UK.

---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org


Mime
View raw message