harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Geir Magnusson Jr <g...@pobox.com>
Subject Re: classlib test suite status emails?
Date Tue, 11 Apr 2006 11:55:32 GMT
So the answer is "no"?

George Harley wrote:
> Geir Magnusson Jr wrote:
>> Just curious (and this isn't a criticism - I'm just as guilty of not 
>> doing this)...
>>
>> Don't you run the tests before committing?
> 
> Hi Geir,
> 
> Depends what do you mean by "the tests".
> 
> The change was completely encapsulated in the text component. I ran the 
> text tests. I checked that the new tests introduced in the issue patch 
> gave the same results when run against a 5.0 RI. All was fine.
> 
> The automated build and test server (which runs all the tests in all the 
> modules) found the problem. That is it's job and it did it well. The 
> problem was found relatively quickly and has now been remedied. The 
> turnaround time would have been quicker if the automated build and test 
> server issued a failure notice on failing tests but that is now in the 
> process of getting sorted out.
> 
> IMHO this problem was a bit of an odd problem as the failures did not 
> arise in the patched module's test suite but in other modules. That is a 
> good indication that the test suite for text is in need of some 
> scrutiny. That is a useful thing to be made aware of.
> 
> 
> Best regards,
> George
> 
>>
>> geir
>>
>> George Harley wrote:
>>> Hi,
>>>
>>> It *seems* like things started failing after I committed the changes 
>>> for HARMONY-205 last night. I'm looking into this now. If the 
>>> investigation begins to take up too much time I will back the changes 
>>> out.
>>>
>>> Best regards,
>>> George
>>>
>>>
>>> Stepan Mishura wrote:
>>>> The same for me + DatagramChannelTest
>>>>
>>>> Thanks,
>>>> Stepan.
>>>>
>>>> On 4/11/06, Mark Hindess wrote:
>>>>
>>>>> No. These:
>>>>>
>>>>> F
>>>>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoder

>>>>>
>>>>> E
>>>>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder01

>>>>>
>>>>> E
>>>>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder02

>>>>>
>>>>> F org.apache.harmony.security.asn1.der.DerUTCTimeEDTest.testMt
>>>>> E
>>>>> org.apache.harmony.security.x509.PrivateKeyUsagePeriodTest.testEncodeDecode

>>>>>
>>>>> F java.security.cert.X509CertSelectorTest.testSetPrivateKeyValid
>>>>> F java.security.cert.X509CertSelectorTest.testMatch
>>>>> F java.security.cert.X509CertSelectorTest.testClone
>>>>> F tests.api.java.sql.DateTest.testSetTimelong
>>>>> F tests.api.java.sql.DateTest.testToString
>>>>> F tests.api.java.sql.DateTest.testValueOf
>>>>> F tests.api.java.sql.TimestampTest.testSetNanosint
>>>>> F tests.api.java.sql.TimestampTest.testToString
>>>>> F tests.api.java.util.DateTest.test_toGMTString
>>>>> F tests.api.java.util.DateTest.test_toString
>>>>>
>>>>> Regards,
>>>>> Mark.
>>>>>
>>>>> On 4/11/06, Stepan Mishura <stepan.mishura@gmail.com> wrote:
>>>>>> On 4/11/06, Mark Hindess wrote:
>>>>>>> Yes. I was using the failureproperty mechanism. Trying to get
this
>>>>>>> property propogated back to the top level ant file was what I
was
>>>>>>> having trouble with.
>>>>>>>
>>>>>>> Using a file as you suggest might help. I'll give that a try
>>>>> shortly...
>>>>>>> Incidentally, I'm seeing 12 failures and 3 errors on r393111.
>>>>>> I guess that you have 9 tests from DatagramChannelTest passed. And

>>>>>> 12 +
>>>>> 3 =
>>>>>> 15 :-)
>>>>>>
>>>>>> (And
>>>>>>> there are typos - "mathc" should be "match" - in the failure

>>>>>>> messages
>>>>>>> for java.security.cert testMatch and testClone.)
>>>>>> I've fixed typo.
>>>>>>
>>>>>> Thanks,
>>>>>> Stepan.
>>>>>>
>>>>>> Regards,
>>>>>>> Mark.
>>>>>>>
>>>>>>> On 4/11/06, Stepan Mishura <stepan.mishura@gmail.com> wrote:
>>>>>>>> On 4/11/06, Mark Hindess wrote:
>>>>>>>>
>>>>>>>>> Stepan,
>>>>>>>>>
>>>>>>>>> I have another build running (but without notifications
going to
>>>>> the
>>>>>>>>> list) that runs:
>>>>>>>>>
>>>>>>>>> 1) build (with reference jdk)
>>>>>>>>> 2) build with what we created with 1)
>>>>>>>>> 3) test
>>>>>>>>> 4) create classlists and compare with class load data
for
>>>>> applications
>>>>>>>>> (tomcat, geronimo, continuum, etc.)
>>>>>>>>> 5) generate JAPI results
>>>>>>>>>
>>>>>>>>> I'd like to fail this build at step 3, but I can't see
an easy way
>>>>> to
>>>>>>>>> get 'ant -f make/build.xml test' to run all tests and
then fail if
>>>>> any
>>>>>>>>> of the module test sub-targets had test failures. I could
parse
>>>>> the
>>>>>>>>> output I suppose, but I'd rather get ant to propagate
the junit
>>>>> tasks
>>>>>>>>> failure property back up to the top level. I've tried
a couple of
>>>>>>>>> things and none seem to work. Any suggestions welcome.
>>>>>>>> Mark, did you try failureproperty parameter for junit task?
>>>>>>>> We may add it to ant sub-targets to raise a flag, for example,
>>>>> create
>>>>>>> file "
>>>>>>>> TESTS.FAILED" in the root, when tests for some module fail.
And in
>>>>> the
>>>>>>> end
>>>>>>>> of tests suite run check whether this file exists on not.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Stepan.
>>>>>>>>
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>>> Mark.
>>>>>>>>>
>>>>>>>>> On 4/11/06, Stepan Mishura <stepan.mishura@gmail.com>
wrote:
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> I've checked out at morning last updates, built the
code base
>>>>> and
>>>>>>> run
>>>>>>>>> the
>>>>>>>>>> tests …and there are 24 tests failures!
>>>>>>>>>>
>>>>>>>>>> There are 9 tests failures in
>>>>>>>>>> org.apache.harmony.tests.java.nio.channels.DatagramChannelTest
–
>>>>> I
>>>>>>> saw
>>>>>>>>> these
>>>>>>>>>> failures before from time to time. It seems that
tests depend on
>>>>>>> some
>>>>>>>>> race
>>>>>>>>>> conditions because they may pass if I rerun them.
Paulex, are
>>>>> these
>>>>>>>>> tests
>>>>>>>>>> passing for you?
>>>>>>>>>>
>>>>>>>>>> And there are new 15 test failures. So now if I'll
make a code
>>>>>>> update
>>>>>>>>> or a
>>>>>>>>>> bug fix how I can be 100% sure that I don't do any
regression?
>>>>>>>>>>
>>>>>>>>>> Currently if a commit breaks the Harmony classlib
build a
>>>>>>> notification
>>>>>>>>> with
>>>>>>>>>> subject: "[continuum] BUILD FAILURE: Classlib/linux.ia32"
will
>>>>> be
>>>>>>> send
>>>>>>>>> to
>>>>>>>>>> harmony-commits mailing list. Is it possible to have
the same
>>>>> for
>>>>>>> tests?
>>>>>>>>> I
>>>>>>>>>> mean that after completing automatic build the existing
classlib
>>>>>>> tests
>>>>>>>>> suite
>>>>>>>>>> should be run. If there are failing tests then a
report
>>>>> notification
>>>>>>>>> with
>>>>>>>>>> corresponding subject will be send.
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Stepan Mishura
>>>>>>>>>> Intel Middleware Products Division
>>>>>>>>> -- 
>>>>>>>>> Mark Hindess <mark.hindess@googlemail.com>
>>>>>>>>> IBM Java Technology Centre, UK.
>>>>>>>>>
>>>>>>>>>
>>>>> ---------------------------------------------------------------------
>>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>>>> To unsubscribe, e-mail:
>>>>> harmony-dev-unsubscribe@incubator.apache.org
>>>>>>>>> For additional commands, e-mail:
>>>>> harmony-dev-help@incubator.apache.org
>>>>>>>> -- 
>>>>>>>>
>>>>> ---------------------------------------------------------------------
>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>>> To unsubscribe, e-mail: 
>>>>>>>> harmony-dev-unsubscribe@incubator.apache.org
>>>>>>>> For additional commands, e-mail:
>>>>> harmony-dev-help@incubator.apache.org
>>>>>>>> Thanks,
>>>>>>>> Stepan Mishura
>>>>>>>> Intel Middleware Products Division
>>>>>>>>
>>>>>>>>
>>>>>>> -- 
>>>>>>> Mark Hindess <mark.hindess@googlemail.com>
>>>>>>> IBM Java Technology Centre, UK.
>>>>>>>
>>>>>>> ---------------------------------------------------------------------

>>>>>>>
>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>>>>> For additional commands, e-mail: 
>>>>>>> harmony-dev-help@incubator.apache.org
>>>>>>>
>>>>>>>
>>>>>> -- 
>>>>>> ---------------------------------------------------------------------
>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>>>> For additional commands, e-mail: 
>>>>>> harmony-dev-help@incubator.apache.org
>>>>>>
>>>>>> Thanks,
>>>>>> Stepan Mishura
>>>>>> Intel Middleware Products Division
>>>>>>
>>>>>>
>>>>> -- 
>>>>> Mark Hindess <mark.hindess@googlemail.com>
>>>>> IBM Java Technology Centre, UK.
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>>>
>>>>>
>>>>
>>>>
>>>> -- 
>>>> ---------------------------------------------------------------------
>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>>
>>>> Thanks,
>>>> Stepan Mishura
>>>> Intel Middleware Products Division
>>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>
>>
> 
> 
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> 
> 

---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org


Mime
View raw message