Return-Path: Delivered-To: apmail-incubator-harmony-dev-archive@www.apache.org Received: (qmail 36899 invoked from network); 11 Apr 2006 10:49:15 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (209.237.227.199) by minotaur.apache.org with SMTP; 11 Apr 2006 10:49:14 -0000 Received: (qmail 70056 invoked by uid 500); 11 Apr 2006 10:49:10 -0000 Delivered-To: apmail-incubator-harmony-dev-archive@incubator.apache.org Received: (qmail 69744 invoked by uid 500); 11 Apr 2006 10:49:09 -0000 Mailing-List: contact harmony-dev-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: harmony-dev@incubator.apache.org Delivered-To: mailing list harmony-dev@incubator.apache.org Received: (qmail 69733 invoked by uid 99); 11 Apr 2006 10:49:09 -0000 Received: from asf.osuosl.org (HELO asf.osuosl.org) (140.211.166.49) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 11 Apr 2006 03:49:09 -0700 X-ASF-Spam-Status: No, hits=1.3 required=10.0 tests=RCVD_IN_BL_SPAMCOP_NET,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (asf.osuosl.org: domain of mark.hindess@googlemail.com designates 64.233.166.178 as permitted sender) Received: from [64.233.166.178] (HELO pproxy.gmail.com) (64.233.166.178) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 11 Apr 2006 03:49:08 -0700 Received: by pproxy.gmail.com with SMTP id o67so2937877pye for ; Tue, 11 Apr 2006 03:48:47 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=googlemail.com; h=received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=EiuZr+6mNG2OBsS8FSf8JVmnxXupggC1Inox9Q+8DT6SBUJRy+ZoTGleXE4n1hhzxUtfnr7sxFWxjzcTebOEN1q8Zb3AH9gFoZr+uXyTG/pWg/t9vAkYp5GcaXoTbYV1WUrMO7+BTsv4y4H7UW7jRgKI1vLwtcEq7shbtDHSAJI= Received: by 10.35.15.11 with SMTP id s11mr1475389pyi; Tue, 11 Apr 2006 03:48:47 -0700 (PDT) Received: by 10.35.63.11 with HTTP; Tue, 11 Apr 2006 03:48:47 -0700 (PDT) Message-ID: Date: Tue, 11 Apr 2006 11:48:47 +0100 From: "Mark Hindess" To: harmony-dev@incubator.apache.org, geir@pobox.com Subject: Re: classlib test suite status emails? In-Reply-To: <443B866E.5050203@pobox.com> MIME-Version: 1.0 Content-Type: text/plain; charset=WINDOWS-1252 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline References: <6e47b64f0604102118p62086311red3af9af86f6c76e@mail.gmail.com> <6e47b64f0604110010j35654822u6b6beaf14666653a@mail.gmail.com> <6e47b64f0604110036i67efd39cob28f3934f0f4e429@mail.gmail.com> <6e47b64f0604110139m55b47003oc55ab2c3c9e4782e@mail.gmail.com> <443B6E2C.6010604@googlemail.com> <443B866E.5050203@pobox.com> X-Virus-Checked: Checked by ClamAV on apache.org X-Spam-Rating: minotaur.apache.org 1.6.2 0/1000/N Personally, obviously, I'd expect people to run the tests before committin= g. However, I notice that since enabling the security tests - which fork for every test - that the tests take over half an hour to run now on our Linux build machine. So I can see why enthusiasm might lead to people not running all the tests but instead perhaps just running those from the module you are changing. George just commented in a commit message backing out some changes in text that all the text tests passed even though the changed caused failures elsewhere. Regards, Mark. On 4/11/06, Geir Magnusson Jr wrote: > Just curious (and this isn't a criticism - I'm just as guilty of not > doing this)... > > Don't you run the tests before committing? > > geir > > George Harley wrote: > > Hi, > > > > It *seems* like things started failing after I committed the changes fo= r > > HARMONY-205 last night. I'm looking into this now. If the investigation > > begins to take up too much time I will back the changes out. > > > > Best regards, > > George > > > > > > Stepan Mishura wrote: > >> The same for me + DatagramChannelTest > >> > >> Thanks, > >> Stepan. > >> > >> On 4/11/06, Mark Hindess wrote: > >> > >>> No. These: > >>> > >>> F > >>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGen= eralizedEncoder > >>> > >>> E > >>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGen= eralizedEncoderDecoder01 > >>> > >>> E > >>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGen= eralizedEncoderDecoder02 > >>> > >>> F org.apache.harmony.security.asn1.der.DerUTCTimeEDTest.testMt > >>> E > >>> org.apache.harmony.security.x509.PrivateKeyUsagePeriodTest.testEncode= Decode > >>> > >>> F java.security.cert.X509CertSelectorTest.testSetPrivateKeyValid > >>> F java.security.cert.X509CertSelectorTest.testMatch > >>> F java.security.cert.X509CertSelectorTest.testClone > >>> F tests.api.java.sql.DateTest.testSetTimelong > >>> F tests.api.java.sql.DateTest.testToString > >>> F tests.api.java.sql.DateTest.testValueOf > >>> F tests.api.java.sql.TimestampTest.testSetNanosint > >>> F tests.api.java.sql.TimestampTest.testToString > >>> F tests.api.java.util.DateTest.test_toGMTString > >>> F tests.api.java.util.DateTest.test_toString > >>> > >>> Regards, > >>> Mark. > >>> > >>> On 4/11/06, Stepan Mishura wrote: > >>> > >>>> On 4/11/06, Mark Hindess wrote: > >>>> > >>>>> Yes. I was using the failureproperty mechanism. Trying to get thi= s > >>>>> property propogated back to the top level ant file was what I was > >>>>> having trouble with. > >>>>> > >>>>> Using a file as you suggest might help. I'll give that a try > >>>>> > >>> shortly... > >>> > >>>>> Incidentally, I'm seeing 12 failures and 3 errors on r393111. > >>>>> > >>>> I guess that you have 9 tests from DatagramChannelTest passed. And > >>>> 12 + > >>>> > >>> 3 =3D > >>> > >>>> 15 :-) > >>>> > >>>> (And > >>>> > >>>>> there are typos - "mathc" should be "match" - in the failure messag= es > >>>>> for java.security.cert testMatch and testClone.) > >>>>> > >>>> I've fixed typo. > >>>> > >>>> Thanks, > >>>> Stepan. > >>>> > >>>> Regards, > >>>> > >>>>> Mark. > >>>>> > >>>>> On 4/11/06, Stepan Mishura wrote: > >>>>> > >>>>>> On 4/11/06, Mark Hindess wrote: > >>>>>> > >>>>>> > >>>>>>> Stepan, > >>>>>>> > >>>>>>> I have another build running (but without notifications going to > >>>>>>> > >>> the > >>> > >>>>>>> list) that runs: > >>>>>>> > >>>>>>> 1) build (with reference jdk) > >>>>>>> 2) build with what we created with 1) > >>>>>>> 3) test > >>>>>>> 4) create classlists and compare with class load data for > >>>>>>> > >>> applications > >>> > >>>>>>> (tomcat, geronimo, continuum, etc.) > >>>>>>> 5) generate JAPI results > >>>>>>> > >>>>>>> I'd like to fail this build at step 3, but I can't see an easy wa= y > >>>>>>> > >>> to > >>> > >>>>>>> get 'ant -f make/build.xml test' to run all tests and then fail i= f > >>>>>>> > >>> any > >>> > >>>>>>> of the module test sub-targets had test failures. I could parse > >>>>>>> > >>> the > >>> > >>>>>>> output I suppose, but I'd rather get ant to propagate the junit > >>>>>>> > >>> tasks > >>> > >>>>>>> failure property back up to the top level. I've tried a couple o= f > >>>>>>> things and none seem to work. Any suggestions welcome. > >>>>>>> > >>>>>> Mark, did you try failureproperty parameter for junit task? > >>>>>> We may add it to ant sub-targets to raise a flag, for example, > >>>>>> > >>> create > >>> > >>>>> file " > >>>>> > >>>>>> TESTS.FAILED" in the root, when tests for some module fail. And i= n > >>>>>> > >>> the > >>> > >>>>> end > >>>>> > >>>>>> of tests suite run check whether this file exists on not. > >>>>>> > >>>>>> Thanks, > >>>>>> Stepan. > >>>>>> > >>>>>> > >>>>>> Regards, > >>>>>> > >>>>>>> Mark. > >>>>>>> > >>>>>>> On 4/11/06, Stepan Mishura wrote: > >>>>>>> > >>>>>>>> Hi, > >>>>>>>> > >>>>>>>> I've checked out at morning last updates, built the code base > >>>>>>>> > >>> and > >>> > >>>>> run > >>>>> > >>>>>>> the > >>>>>>> > >>>>>>>> tests =85and there are 24 tests failures! > >>>>>>>> > >>>>>>>> There are 9 tests failures in > >>>>>>>> org.apache.harmony.tests.java.nio.channels.DatagramChannelTest = =96 > >>>>>>>> > >>> I > >>> > >>>>> saw > >>>>> > >>>>>>> these > >>>>>>> > >>>>>>>> failures before from time to time. It seems that tests depend on > >>>>>>>> > >>>>> some > >>>>> > >>>>>>> race > >>>>>>> > >>>>>>>> conditions because they may pass if I rerun them. Paulex, are > >>>>>>>> > >>> these > >>> > >>>>>>> tests > >>>>>>> > >>>>>>>> passing for you? > >>>>>>>> > >>>>>>>> And there are new 15 test failures. So now if I'll make a code > >>>>>>>> > >>>>> update > >>>>> > >>>>>>> or a > >>>>>>> > >>>>>>>> bug fix how I can be 100% sure that I don't do any regression? > >>>>>>>> > >>>>>>>> Currently if a commit breaks the Harmony classlib build a > >>>>>>>> > >>>>> notification > >>>>> > >>>>>>> with > >>>>>>> > >>>>>>>> subject: "[continuum] BUILD FAILURE: Classlib/linux.ia32" will > >>>>>>>> > >>> be > >>> > >>>>> send > >>>>> > >>>>>>> to > >>>>>>> > >>>>>>>> harmony-commits mailing list. Is it possible to have the same > >>>>>>>> > >>> for > >>> > >>>>> tests? > >>>>> > >>>>>>> I > >>>>>>> > >>>>>>>> mean that after completing automatic build the existing classlib > >>>>>>>> > >>>>> tests > >>>>> > >>>>>>> suite > >>>>>>> > >>>>>>>> should be run. If there are failing tests then a report > >>>>>>>> > >>> notification > >>> > >>>>>>> with > >>>>>>> > >>>>>>>> corresponding subject will be send. > >>>>>>>> > >>>>>>>> Thanks, > >>>>>>>> Stepan Mishura > >>>>>>>> Intel Middleware Products Division > >>>>>>>> > >>>>>>> -- > >>>>>>> Mark Hindess > >>>>>>> IBM Java Technology Centre, UK. > >>>>>>> > >>>>>>> > >>>>>>> > >>> --------------------------------------------------------------------- > >>> > >>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html > >>>>>>> To unsubscribe, e-mail: > >>>>>>> > >>> harmony-dev-unsubscribe@incubator.apache.org > >>> > >>>>>>> For additional commands, e-mail: > >>>>>>> > >>> harmony-dev-help@incubator.apache.org > >>> > >>>>>>> > >>>>>> -- > >>>>>> > >>>>>> > >>> --------------------------------------------------------------------- > >>> > >>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html > >>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.o= rg > >>>>>> For additional commands, e-mail: > >>>>>> > >>> harmony-dev-help@incubator.apache.org > >>> > >>>>>> Thanks, > >>>>>> Stepan Mishura > >>>>>> Intel Middleware Products Division > >>>>>> > >>>>>> > >>>>>> > >>>>> -- > >>>>> Mark Hindess > >>>>> IBM Java Technology Centre, UK. > >>>>> > >>>>> -------------------------------------------------------------------= -- > >>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html > >>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.or= g > >>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.= org > >>>>> > >>>>> > >>>>> > >>>> -- > >>>> --------------------------------------------------------------------= - > >>>> Terms of use : http://incubator.apache.org/harmony/mailing.html > >>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org > >>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.o= rg > >>>> > >>>> Thanks, > >>>> Stepan Mishura > >>>> Intel Middleware Products Division > >>>> > >>>> > >>>> > >>> -- > >>> Mark Hindess > >>> IBM Java Technology Centre, UK. > >>> > >>> --------------------------------------------------------------------- > >>> Terms of use : http://incubator.apache.org/harmony/mailing.html > >>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org > >>> For additional commands, e-mail: harmony-dev-help@incubator.apache.or= g > >>> > >>> > >>> > >> > >> > >> -- > >> --------------------------------------------------------------------- > >> Terms of use : http://incubator.apache.org/harmony/mailing.html > >> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org > >> For additional commands, e-mail: harmony-dev-help@incubator.apache.org > >> > >> Thanks, > >> Stepan Mishura > >> Intel Middleware Products Division > >> > >> > > > > > > --------------------------------------------------------------------- > > Terms of use : http://incubator.apache.org/harmony/mailing.html > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org > > For additional commands, e-mail: harmony-dev-help@incubator.apache.org > > > > > > --------------------------------------------------------------------- > Terms of use : http://incubator.apache.org/harmony/mailing.html > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org > For additional commands, e-mail: harmony-dev-help@incubator.apache.org > > -- Mark Hindess IBM Java Technology Centre, UK. --------------------------------------------------------------------- Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org For additional commands, e-mail: harmony-dev-help@incubator.apache.org