Return-Path: Delivered-To: apmail-incubator-harmony-dev-archive@www.apache.org Received: (qmail 6879 invoked from network); 11 Apr 2006 12:38:12 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (209.237.227.199) by minotaur.apache.org with SMTP; 11 Apr 2006 12:38:10 -0000 Received: (qmail 24522 invoked by uid 500); 11 Apr 2006 12:37:50 -0000 Delivered-To: apmail-incubator-harmony-dev-archive@incubator.apache.org Received: (qmail 24463 invoked by uid 500); 11 Apr 2006 12:37:50 -0000 Mailing-List: contact harmony-dev-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: harmony-dev@incubator.apache.org Delivered-To: mailing list harmony-dev@incubator.apache.org Received: (qmail 24425 invoked by uid 99); 11 Apr 2006 12:37:50 -0000 Received: from asf.osuosl.org (HELO asf.osuosl.org) (140.211.166.49) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 11 Apr 2006 05:37:50 -0700 X-ASF-Spam-Status: No, hits=1.4 required=10.0 tests=SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (asf.osuosl.org: 64.74.244.71 is neither permitted nor denied by domain of geir@pobox.com) Received: from [64.74.244.71] (HELO chi.mobile-health-diary.com) (64.74.244.71) by apache.org (qpsmtpd/0.29) with SMTP; Tue, 11 Apr 2006 05:37:48 -0700 Received: (qmail 11702 invoked from network); 11 Apr 2006 12:37:25 -0000 Received: from ool-43560634.dyn.optonline.net (HELO ?192.168.1.100?) (geir@67.86.6.52) by b014.internal.mobile-health-diary.com with SMTP; 11 Apr 2006 12:37:25 -0000 Message-ID: <443BA27D.2040509@pobox.com> Date: Tue, 11 Apr 2006 08:35:09 -0400 From: Geir Magnusson Jr Reply-To: geir@pobox.com User-Agent: Thunderbird 1.5 (Windows/20051201) MIME-Version: 1.0 To: harmony-dev@incubator.apache.org Subject: Re: classlib test suite status emails? References: <6e47b64f0604102118p62086311red3af9af86f6c76e@mail.gmail.com> <6e47b64f0604110010j35654822u6b6beaf14666653a@mail.gmail.com> <6e47b64f0604110036i67efd39cob28f3934f0f4e429@mail.gmail.com> <6e47b64f0604110139m55b47003oc55ab2c3c9e4782e@mail.gmail.com> <443B6E2C.6010604@googlemail.com> <443B866E.5050203@pobox.com> <443B8A71.8080804@googlemail.com> <443B9934.3010702@pobox.com> <443B9CC3.7050007@googlemail.com> In-Reply-To: <443B9CC3.7050007@googlemail.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit X-Virus-Checked: Checked by ClamAV on apache.org X-Spam-Rating: minotaur.apache.org 1.6.2 0/1000/N George Harley wrote: > Hi Geir, > > From my point of view the answer is yes, I ran the tests for the > patched module. The build/test server picked up the downstream breakages > in the other modules. I fixed matters so that the builds ran again. Right - I thought we considered the build/test server a safety to keep accidents from getting away from us, but given the delay and the distribution of people throughout the planet working on this (rather than all sitting in the same office), we want to avoid breakages at all costs. (Again, I'm guilty of the same thing...) > > Elsewhere in this thread I am trying to cooperate with others to make it > simpler to run more - just the necessary - tests prior to making a commit. I guess the question is "what is necessary"? Given a dependency chain A <- B <- C I don't think it's only the immediate dependencies of a given module A, but rather the full dependency graph, because a "1 away" module (C) might use a direct dependency (B) in a way different than the test cases for that direct dependency do (especially since we aren't quite up to speed with doing implementation testing...) > > Thank you for your support. > Um... geir > > Best regards, > George > > > Geir Magnusson Jr wrote: >> So the answer is "no"? >> >> George Harley wrote: >>> Geir Magnusson Jr wrote: >>>> Just curious (and this isn't a criticism - I'm just as guilty of not >>>> doing this)... >>>> >>>> Don't you run the tests before committing? >>> >>> Hi Geir, >>> >>> Depends what do you mean by "the tests". >>> >>> The change was completely encapsulated in the text component. I ran >>> the text tests. I checked that the new tests introduced in the issue >>> patch gave the same results when run against a 5.0 RI. All was fine. >>> >>> The automated build and test server (which runs all the tests in all >>> the modules) found the problem. That is it's job and it did it well. >>> The problem was found relatively quickly and has now been remedied. >>> The turnaround time would have been quicker if the automated build >>> and test server issued a failure notice on failing tests but that is >>> now in the process of getting sorted out. >>> >>> IMHO this problem was a bit of an odd problem as the failures did not >>> arise in the patched module's test suite but in other modules. That >>> is a good indication that the test suite for text is in need of some >>> scrutiny. That is a useful thing to be made aware of. >>> >>> >>> Best regards, >>> George >>> >>>> >>>> geir >>>> >>>> George Harley wrote: >>>>> Hi, >>>>> >>>>> It *seems* like things started failing after I committed the >>>>> changes for HARMONY-205 last night. I'm looking into this now. If >>>>> the investigation begins to take up too much time I will back the >>>>> changes out. >>>>> >>>>> Best regards, >>>>> George >>>>> >>>>> >>>>> Stepan Mishura wrote: >>>>>> The same for me + DatagramChannelTest >>>>>> >>>>>> Thanks, >>>>>> Stepan. >>>>>> >>>>>> On 4/11/06, Mark Hindess wrote: >>>>>> >>>>>>> No. These: >>>>>>> >>>>>>> F >>>>>>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoder >>>>>>> >>>>>>> E >>>>>>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder01 >>>>>>> >>>>>>> E >>>>>>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder02 >>>>>>> >>>>>>> F org.apache.harmony.security.asn1.der.DerUTCTimeEDTest.testMt >>>>>>> E >>>>>>> org.apache.harmony.security.x509.PrivateKeyUsagePeriodTest.testEncodeDecode >>>>>>> >>>>>>> F java.security.cert.X509CertSelectorTest.testSetPrivateKeyValid >>>>>>> F java.security.cert.X509CertSelectorTest.testMatch >>>>>>> F java.security.cert.X509CertSelectorTest.testClone >>>>>>> F tests.api.java.sql.DateTest.testSetTimelong >>>>>>> F tests.api.java.sql.DateTest.testToString >>>>>>> F tests.api.java.sql.DateTest.testValueOf >>>>>>> F tests.api.java.sql.TimestampTest.testSetNanosint >>>>>>> F tests.api.java.sql.TimestampTest.testToString >>>>>>> F tests.api.java.util.DateTest.test_toGMTString >>>>>>> F tests.api.java.util.DateTest.test_toString >>>>>>> >>>>>>> Regards, >>>>>>> Mark. >>>>>>> >>>>>>> On 4/11/06, Stepan Mishura wrote: >>>>>>>> On 4/11/06, Mark Hindess wrote: >>>>>>>>> Yes. I was using the failureproperty mechanism. Trying to get this >>>>>>>>> property propogated back to the top level ant file was what I was >>>>>>>>> having trouble with. >>>>>>>>> >>>>>>>>> Using a file as you suggest might help. I'll give that a try >>>>>>> shortly... >>>>>>>>> Incidentally, I'm seeing 12 failures and 3 errors on r393111. >>>>>>>> I guess that you have 9 tests from DatagramChannelTest passed. >>>>>>>> And 12 + >>>>>>> 3 = >>>>>>>> 15 :-) >>>>>>>> >>>>>>>> (And >>>>>>>>> there are typos - "mathc" should be "match" - in the failure >>>>>>>>> messages >>>>>>>>> for java.security.cert testMatch and testClone.) >>>>>>>> I've fixed typo. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Stepan. >>>>>>>> >>>>>>>> Regards, >>>>>>>>> Mark. >>>>>>>>> >>>>>>>>> On 4/11/06, Stepan Mishura wrote: >>>>>>>>>> On 4/11/06, Mark Hindess wrote: >>>>>>>>>> >>>>>>>>>>> Stepan, >>>>>>>>>>> >>>>>>>>>>> I have another build running (but without notifications going to >>>>>>> the >>>>>>>>>>> list) that runs: >>>>>>>>>>> >>>>>>>>>>> 1) build (with reference jdk) >>>>>>>>>>> 2) build with what we created with 1) >>>>>>>>>>> 3) test >>>>>>>>>>> 4) create classlists and compare with class load data for >>>>>>> applications >>>>>>>>>>> (tomcat, geronimo, continuum, etc.) >>>>>>>>>>> 5) generate JAPI results >>>>>>>>>>> >>>>>>>>>>> I'd like to fail this build at step 3, but I can't see an >>>>>>>>>>> easy way >>>>>>> to >>>>>>>>>>> get 'ant -f make/build.xml test' to run all tests and then >>>>>>>>>>> fail if >>>>>>> any >>>>>>>>>>> of the module test sub-targets had test failures. I could parse >>>>>>> the >>>>>>>>>>> output I suppose, but I'd rather get ant to propagate the junit >>>>>>> tasks >>>>>>>>>>> failure property back up to the top level. I've tried a >>>>>>>>>>> couple of >>>>>>>>>>> things and none seem to work. Any suggestions welcome. >>>>>>>>>> Mark, did you try failureproperty parameter for junit task? >>>>>>>>>> We may add it to ant sub-targets to raise a flag, for example, >>>>>>> create >>>>>>>>> file " >>>>>>>>>> TESTS.FAILED" in the root, when tests for some module fail. >>>>>>>>>> And in >>>>>>> the >>>>>>>>> end >>>>>>>>>> of tests suite run check whether this file exists on not. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Stepan. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Regards, >>>>>>>>>>> Mark. >>>>>>>>>>> >>>>>>>>>>> On 4/11/06, Stepan Mishura wrote: >>>>>>>>>>>> Hi, >>>>>>>>>>>> >>>>>>>>>>>> I've checked out at morning last updates, built the code base >>>>>>> and >>>>>>>>> run >>>>>>>>>>> the >>>>>>>>>>>> tests �and there are 24 tests failures! >>>>>>>>>>>> >>>>>>>>>>>> There are 9 tests failures in >>>>>>>>>>>> org.apache.harmony.tests.java.nio.channels.DatagramChannelTest >>>>>>>>>>>> � >>>>>>> I >>>>>>>>> saw >>>>>>>>>>> these >>>>>>>>>>>> failures before from time to time. It seems that tests >>>>>>>>>>>> depend on >>>>>>>>> some >>>>>>>>>>> race >>>>>>>>>>>> conditions because they may pass if I rerun them. Paulex, are >>>>>>> these >>>>>>>>>>> tests >>>>>>>>>>>> passing for you? >>>>>>>>>>>> >>>>>>>>>>>> And there are new 15 test failures. So now if I'll make a code >>>>>>>>> update >>>>>>>>>>> or a >>>>>>>>>>>> bug fix how I can be 100% sure that I don't do any regression? >>>>>>>>>>>> >>>>>>>>>>>> Currently if a commit breaks the Harmony classlib build a >>>>>>>>> notification >>>>>>>>>>> with >>>>>>>>>>>> subject: "[continuum] BUILD FAILURE: Classlib/linux.ia32" will >>>>>>> be >>>>>>>>> send >>>>>>>>>>> to >>>>>>>>>>>> harmony-commits mailing list. Is it possible to have the same >>>>>>> for >>>>>>>>> tests? >>>>>>>>>>> I >>>>>>>>>>>> mean that after completing automatic build the existing >>>>>>>>>>>> classlib >>>>>>>>> tests >>>>>>>>>>> suite >>>>>>>>>>>> should be run. If there are failing tests then a report >>>>>>> notification >>>>>>>>>>> with >>>>>>>>>>>> corresponding subject will be send. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Stepan Mishura >>>>>>>>>>>> Intel Middleware Products Division >>>>>>>>>>> -- >>>>>>>>>>> Mark Hindess >>>>>>>>>>> IBM Java Technology Centre, UK. >>>>>>>>>>> >>>>>>>>>>> >>>>>>> --------------------------------------------------------------------- >>>>>>> >>>>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html >>>>>>>>>>> To unsubscribe, e-mail: >>>>>>> harmony-dev-unsubscribe@incubator.apache.org >>>>>>>>>>> For additional commands, e-mail: >>>>>>> harmony-dev-help@incubator.apache.org >>>>>>>>>> -- >>>>>>>>>> >>>>>>> --------------------------------------------------------------------- >>>>>>> >>>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html >>>>>>>>>> To unsubscribe, e-mail: >>>>>>>>>> harmony-dev-unsubscribe@incubator.apache.org >>>>>>>>>> For additional commands, e-mail: >>>>>>> harmony-dev-help@incubator.apache.org >>>>>>>>>> Thanks, >>>>>>>>>> Stepan Mishura >>>>>>>>>> Intel Middleware Products Division >>>>>>>>>> >>>>>>>>>> >>>>>>>>> -- >>>>>>>>> Mark Hindess >>>>>>>>> IBM Java Technology Centre, UK. >>>>>>>>> >>>>>>>>> --------------------------------------------------------------------- >>>>>>>>> >>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html >>>>>>>>> To unsubscribe, e-mail: >>>>>>>>> harmony-dev-unsubscribe@incubator.apache.org >>>>>>>>> For additional commands, e-mail: >>>>>>>>> harmony-dev-help@incubator.apache.org >>>>>>>>> >>>>>>>>> >>>>>>>> -- >>>>>>>> --------------------------------------------------------------------- >>>>>>>> >>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html >>>>>>>> To unsubscribe, e-mail: >>>>>>>> harmony-dev-unsubscribe@incubator.apache.org >>>>>>>> For additional commands, e-mail: >>>>>>>> harmony-dev-help@incubator.apache.org >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Stepan Mishura >>>>>>>> Intel Middleware Products Division >>>>>>>> >>>>>>>> >>>>>>> -- >>>>>>> Mark Hindess >>>>>>> IBM Java Technology Centre, UK. >>>>>>> >>>>>>> --------------------------------------------------------------------- >>>>>>> >>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html >>>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org >>>>>>> For additional commands, e-mail: >>>>>>> harmony-dev-help@incubator.apache.org >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> --------------------------------------------------------------------- >>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html >>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org >>>>>> For additional commands, e-mail: >>>>>> harmony-dev-help@incubator.apache.org >>>>>> >>>>>> Thanks, >>>>>> Stepan Mishura >>>>>> Intel Middleware Products Division >>>>>> >>>>> >>>>> >>>>> --------------------------------------------------------------------- >>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html >>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org >>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org >>>>> >>>>> >>>> >>>> --------------------------------------------------------------------- >>>> Terms of use : http://incubator.apache.org/harmony/mailing.html >>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org >>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org >>>> >>>> >>> >>> >>> --------------------------------------------------------------------- >>> Terms of use : http://incubator.apache.org/harmony/mailing.html >>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org >>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org >>> >>> >> >> --------------------------------------------------------------------- >> Terms of use : http://incubator.apache.org/harmony/mailing.html >> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org >> For additional commands, e-mail: harmony-dev-help@incubator.apache.org >> >> > > > --------------------------------------------------------------------- > Terms of use : http://incubator.apache.org/harmony/mailing.html > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org > For additional commands, e-mail: harmony-dev-help@incubator.apache.org > > --------------------------------------------------------------------- Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org For additional commands, e-mail: harmony-dev-help@incubator.apache.org