Return-Path: Delivered-To: apmail-incubator-harmony-dev-archive@www.apache.org Received: (qmail 54388 invoked from network); 11 Apr 2006 11:20:33 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (209.237.227.199) by minotaur.apache.org with SMTP; 11 Apr 2006 11:20:33 -0000 Received: (qmail 21364 invoked by uid 500); 11 Apr 2006 11:20:23 -0000 Delivered-To: apmail-incubator-harmony-dev-archive@incubator.apache.org Received: (qmail 21313 invoked by uid 500); 11 Apr 2006 11:20:23 -0000 Mailing-List: contact harmony-dev-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: harmony-dev@incubator.apache.org Delivered-To: mailing list harmony-dev@incubator.apache.org Received: (qmail 21302 invoked by uid 99); 11 Apr 2006 11:20:23 -0000 Received: from asf.osuosl.org (HELO asf.osuosl.org) (140.211.166.49) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 11 Apr 2006 04:20:23 -0700 X-ASF-Spam-Status: No, hits=0.0 required=10.0 tests=HTML_MESSAGE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (asf.osuosl.org: domain of stepan.mishura@gmail.com designates 64.233.182.191 as permitted sender) Received: from [64.233.182.191] (HELO nproxy.gmail.com) (64.233.182.191) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 11 Apr 2006 04:20:21 -0700 Received: by nproxy.gmail.com with SMTP id l36so768218nfa for ; Tue, 11 Apr 2006 04:20:00 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:references; b=QQtSKjjPJNO18HatTPNRnWFqQo1lYQbgGx1jxEmwVIo/PpC5Pozc+6mbzpcjqgFSx/Qs7WmecZb+8y1pKsrdd8a2aiFIm0QoFufX6O9GPe0JtCMZVc6zcSgxzBFP02AsiGEUWAiwDG2D70PMSIJrbc3Sm8zAwL16lteZjH0o+vU= Received: by 10.49.4.10 with SMTP id g10mr4446844nfi; Tue, 11 Apr 2006 04:19:59 -0700 (PDT) Received: by 10.49.30.1 with HTTP; Tue, 11 Apr 2006 04:19:59 -0700 (PDT) Message-ID: <6e47b64f0604110419k441ae66as92c3e0c855533eb@mail.gmail.com> Date: Tue, 11 Apr 2006 18:19:59 +0700 From: "Stepan Mishura" To: harmony-dev@incubator.apache.org Subject: Re: classlib test suite status emails? In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_57332_30177098.1144754399915" References: <6e47b64f0604102118p62086311red3af9af86f6c76e@mail.gmail.com> <6e47b64f0604110010j35654822u6b6beaf14666653a@mail.gmail.com> <6e47b64f0604110036i67efd39cob28f3934f0f4e429@mail.gmail.com> <6e47b64f0604110139m55b47003oc55ab2c3c9e4782e@mail.gmail.com> <443B6E2C.6010604@googlemail.com> <443B866E.5050203@pobox.com> X-Virus-Checked: Checked by ClamAV on apache.org X-Spam-Rating: minotaur.apache.org 1.6.2 0/1000/N ------=_Part_57332_30177098.1144754399915 Content-Type: text/plain; charset=WINDOWS-1252 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline On 4/11/06, Mark Hindess wrote: > Personally, obviously, I'd expect people to run the tests before > committing. > > However, I notice that since enabling the security tests - which fork > for every test - that the tests take over half an hour to run now on > our Linux build machine. So I can see why enthusiasm might lead to > people not running all the tests but instead perhaps just running > those from the module you are changing. I think that this message[1] relates to the problem with running big test suites. So should we do the similar thing i.e. define which test suites should be run for each module? Thanks, Stepan. [1] http://mail-archives.apache.org/mod_mbox/incubator-harmony-dev/200603.mbox/= %3c442C1C79.1060805@sourcelabs.com%3e > George just commented in a commit message backing out some changes in > text that all the text tests passed even though the changed caused > failures elsewhere. > > Regards, > Mark. > > On 4/11/06, Geir Magnusson Jr wrote: > > Just curious (and this isn't a criticism - I'm just as guilty of not > > doing this)... > > > > Don't you run the tests before committing? > > > > geir > > > > George Harley wrote: > > > Hi, > > > > > > It *seems* like things started failing after I committed the changes > for > > > HARMONY-205 last night. I'm looking into this now. If the > investigation > > > begins to take up too much time I will back the changes out. > > > > > > Best regards, > > > George > > > > > > > > > Stepan Mishura wrote: > > >> The same for me + DatagramChannelTest > > >> > > >> Thanks, > > >> Stepan. > > >> > > >> On 4/11/06, Mark Hindess wrote: > > >> > > >>> No. These: > > >>> > > >>> F > > >>> > org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneral= izedEncoder > > >>> > > >>> E > > >>> > org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneral= izedEncoderDecoder01 > > >>> > > >>> E > > >>> > org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneral= izedEncoderDecoder02 > > >>> > > >>> F org.apache.harmony.security.asn1.der.DerUTCTimeEDTest.testMt > > >>> E > > >>> > org.apache.harmony.security.x509.PrivateKeyUsagePeriodTest.testEncodeDeco= de > > >>> > > >>> F java.security.cert.X509CertSelectorTest.testSetPrivateKeyValid > > >>> F java.security.cert.X509CertSelectorTest.testMatch > > >>> F java.security.cert.X509CertSelectorTest.testClone > > >>> F tests.api.java.sql.DateTest.testSetTimelong > > >>> F tests.api.java.sql.DateTest.testToString > > >>> F tests.api.java.sql.DateTest.testValueOf > > >>> F tests.api.java.sql.TimestampTest.testSetNanosint > > >>> F tests.api.java.sql.TimestampTest.testToString > > >>> F tests.api.java.util.DateTest.test_toGMTString > > >>> F tests.api.java.util.DateTest.test_toString > > >>> > > >>> Regards, > > >>> Mark. > > >>> > > >>> On 4/11/06, Stepan Mishura wrote: > > >>> > > >>>> On 4/11/06, Mark Hindess wrote: > > >>>> > > >>>>> Yes. I was using the failureproperty mechanism. Trying to get > this > > >>>>> property propogated back to the top level ant file was what I was > > >>>>> having trouble with. > > >>>>> > > >>>>> Using a file as you suggest might help. I'll give that a try > > >>>>> > > >>> shortly... > > >>> > > >>>>> Incidentally, I'm seeing 12 failures and 3 errors on r393111. > > >>>>> > > >>>> I guess that you have 9 tests from DatagramChannelTest passed. An= d > > >>>> 12 + > > >>>> > > >>> 3 =3D > > >>> > > >>>> 15 :-) > > >>>> > > >>>> (And > > >>>> > > >>>>> there are typos - "mathc" should be "match" - in the failure > messages > > >>>>> for java.security.cert testMatch and testClone.) > > >>>>> > > >>>> I've fixed typo. > > >>>> > > >>>> Thanks, > > >>>> Stepan. > > >>>> > > >>>> Regards, > > >>>> > > >>>>> Mark. > > >>>>> > > >>>>> On 4/11/06, Stepan Mishura wrote: > > >>>>> > > >>>>>> On 4/11/06, Mark Hindess wrote: > > >>>>>> > > >>>>>> > > >>>>>>> Stepan, > > >>>>>>> > > >>>>>>> I have another build running (but without notifications going t= o > > >>>>>>> > > >>> the > > >>> > > >>>>>>> list) that runs: > > >>>>>>> > > >>>>>>> 1) build (with reference jdk) > > >>>>>>> 2) build with what we created with 1) > > >>>>>>> 3) test > > >>>>>>> 4) create classlists and compare with class load data for > > >>>>>>> > > >>> applications > > >>> > > >>>>>>> (tomcat, geronimo, continuum, etc.) > > >>>>>>> 5) generate JAPI results > > >>>>>>> > > >>>>>>> I'd like to fail this build at step 3, but I can't see an easy > way > > >>>>>>> > > >>> to > > >>> > > >>>>>>> get 'ant -f make/build.xml test' to run all tests and then fail > if > > >>>>>>> > > >>> any > > >>> > > >>>>>>> of the module test sub-targets had test failures. I could pars= e > > >>>>>>> > > >>> the > > >>> > > >>>>>>> output I suppose, but I'd rather get ant to propagate the junit > > >>>>>>> > > >>> tasks > > >>> > > >>>>>>> failure property back up to the top level. I've tried a couple > of > > >>>>>>> things and none seem to work. Any suggestions welcome. > > >>>>>>> > > >>>>>> Mark, did you try failureproperty parameter for junit task? > > >>>>>> We may add it to ant sub-targets to raise a flag, for example, > > >>>>>> > > >>> create > > >>> > > >>>>> file " > > >>>>> > > >>>>>> TESTS.FAILED" in the root, when tests for some module fail. And > in > > >>>>>> > > >>> the > > >>> > > >>>>> end > > >>>>> > > >>>>>> of tests suite run check whether this file exists on not. > > >>>>>> > > >>>>>> Thanks, > > >>>>>> Stepan. > > >>>>>> > > >>>>>> > > >>>>>> Regards, > > >>>>>> > > >>>>>>> Mark. > > >>>>>>> > > >>>>>>> On 4/11/06, Stepan Mishura wrote: > > >>>>>>> > > >>>>>>>> Hi, > > >>>>>>>> > > >>>>>>>> I've checked out at morning last updates, built the code base > > >>>>>>>> > > >>> and > > >>> > > >>>>> run > > >>>>> > > >>>>>>> the > > >>>>>>> > > >>>>>>>> tests =85and there are 24 tests failures! > > >>>>>>>> > > >>>>>>>> There are 9 tests failures in > > >>>>>>>> org.apache.harmony.tests.java.nio.channels.DatagramChannelTest= =96 > > >>>>>>>> > > >>> I > > >>> > > >>>>> saw > > >>>>> > > >>>>>>> these > > >>>>>>> > > >>>>>>>> failures before from time to time. It seems that tests depend > on > > >>>>>>>> > > >>>>> some > > >>>>> > > >>>>>>> race > > >>>>>>> > > >>>>>>>> conditions because they may pass if I rerun them. Paulex, are > > >>>>>>>> > > >>> these > > >>> > > >>>>>>> tests > > >>>>>>> > > >>>>>>>> passing for you? > > >>>>>>>> > > >>>>>>>> And there are new 15 test failures. So now if I'll make a cod= e > > >>>>>>>> > > >>>>> update > > >>>>> > > >>>>>>> or a > > >>>>>>> > > >>>>>>>> bug fix how I can be 100% sure that I don't do any regression? > > >>>>>>>> > > >>>>>>>> Currently if a commit breaks the Harmony classlib build a > > >>>>>>>> > > >>>>> notification > > >>>>> > > >>>>>>> with > > >>>>>>> > > >>>>>>>> subject: "[continuum] BUILD FAILURE: Classlib/linux.ia32" will > > >>>>>>>> > > >>> be > > >>> > > >>>>> send > > >>>>> > > >>>>>>> to > > >>>>>>> > > >>>>>>>> harmony-commits mailing list. Is it possible to have the same > > >>>>>>>> > > >>> for > > >>> > > >>>>> tests? > > >>>>> > > >>>>>>> I > > >>>>>>> > > >>>>>>>> mean that after completing automatic build the existing > classlib > > >>>>>>>> > > >>>>> tests > > >>>>> > > >>>>>>> suite > > >>>>>>> > > >>>>>>>> should be run. If there are failing tests then a report > > >>>>>>>> > > >>> notification > > >>> > > >>>>>>> with > > >>>>>>> > > >>>>>>>> corresponding subject will be send. > > >>>>>>>> > > >>>>>>>> Thanks, > > >>>>>>>> Stepan Mishura > > >>>>>>>> Intel Middleware Products Division > > >>>>>>>> > > >>>>>>> -- > > >>>>>>> Mark Hindess > > >>>>>>> IBM Java Technology Centre, UK. > > >>>>>>> > > >>>>>>> > > >>>>>>> > > >>> > --------------------------------------------------------------------- > > >>> > > >>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html > > >>>>>>> To unsubscribe, e-mail: > > >>>>>>> > > >>> harmony-dev-unsubscribe@incubator.apache.org > > >>> > > >>>>>>> For additional commands, e-mail: > > >>>>>>> > > >>> harmony-dev-help@incubator.apache.org > > >>> > > >>>>>>> > > >>>>>> -- > > >>>>>> > > >>>>>> > > >>> > --------------------------------------------------------------------- > > >>> > > >>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html > > >>>>>> To unsubscribe, e-mail: > harmony-dev-unsubscribe@incubator.apache.org > > >>>>>> For additional commands, e-mail: > > >>>>>> > > >>> harmony-dev-help@incubator.apache.org > > >>> > > >>>>>> Thanks, > > >>>>>> Stepan Mishura > > >>>>>> Intel Middleware Products Division > > >>>>>> > > >>>>>> > > >>>>>> > > >>>>> -- > > >>>>> Mark Hindess > > >>>>> IBM Java Technology Centre, UK. > > >>>>> > > >>>>> > --------------------------------------------------------------------- > > >>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html > > >>>>> To unsubscribe, e-mail: > harmony-dev-unsubscribe@incubator.apache.org > > >>>>> For additional commands, e-mail: > harmony-dev-help@incubator.apache.org > > >>>>> > > >>>>> > > >>>>> > > >>>> -- > > >>>> > --------------------------------------------------------------------- > > >>>> Terms of use : http://incubator.apache.org/harmony/mailing.html > > >>>> To unsubscribe, e-mail: > harmony-dev-unsubscribe@incubator.apache.org > > >>>> For additional commands, e-mail: > harmony-dev-help@incubator.apache.org > > >>>> > > >>>> Thanks, > > >>>> Stepan Mishura > > >>>> Intel Middleware Products Division > > >>>> > > >>>> > > >>>> > > >>> -- > > >>> Mark Hindess > > >>> IBM Java Technology Centre, UK. > > >>> > > >>> > --------------------------------------------------------------------- > > >>> Terms of use : http://incubator.apache.org/harmony/mailing.html > > >>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.or= g > > >>> For additional commands, e-mail: > harmony-dev-help@incubator.apache.org > > >>> > > >>> > > >>> > > >> > > >> > > >> -- > > >> --------------------------------------------------------------------= - > > >> Terms of use : http://incubator.apache.org/harmony/mailing.html > > >> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org > > >> For additional commands, e-mail: > harmony-dev-help@incubator.apache.org > > >> > > >> Thanks, > > >> Stepan Mishura > > >> Intel Middleware Products Division > > >> > > >> > > > > > > > > > --------------------------------------------------------------------- > > > Terms of use : http://incubator.apache.org/harmony/mailing.html > > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org > > > For additional commands, e-mail: harmony-dev-help@incubator.apache.or= g > > > > > > > > > > --------------------------------------------------------------------- > > Terms of use : http://incubator.apache.org/harmony/mailing.html > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org > > For additional commands, e-mail: harmony-dev-help@incubator.apache.org > > > > > > > -- > Mark Hindess > IBM Java Technology Centre, UK. > > --------------------------------------------------------------------- > Terms of use : http://incubator.apache.org/harmony/mailing.html > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org > For additional commands, e-mail: harmony-dev-help@incubator.apache.org > > -- --------------------------------------------------------------------- Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org For additional commands, e-mail: harmony-dev-help@incubator.apache.org Thanks, Stepan Mishura Intel Middleware Products Division ------=_Part_57332_30177098.1144754399915--