harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From George Harley <george.c.har...@googlemail.com>
Subject Re: classlib test suite status emails?
Date Tue, 11 Apr 2006 10:52:33 GMT
Geir Magnusson Jr wrote:
> Just curious (and this isn't a criticism - I'm just as guilty of not 
> doing this)...
>
> Don't you run the tests before committing?

Hi Geir,

Depends what do you mean by "the tests".

The change was completely encapsulated in the text component. I ran the 
text tests. I checked that the new tests introduced in the issue patch 
gave the same results when run against a 5.0 RI. All was fine.

The automated build and test server (which runs all the tests in all the 
modules) found the problem. That is it's job and it did it well. The 
problem was found relatively quickly and has now been remedied. The 
turnaround time would have been quicker if the automated build and test 
server issued a failure notice on failing tests but that is now in the 
process of getting sorted out.

IMHO this problem was a bit of an odd problem as the failures did not 
arise in the patched module's test suite but in other modules. That is a 
good indication that the test suite for text is in need of some 
scrutiny. That is a useful thing to be made aware of.


Best regards,
George

>
> geir
>
> George Harley wrote:
>> Hi,
>>
>> It *seems* like things started failing after I committed the changes 
>> for HARMONY-205 last night. I'm looking into this now. If the 
>> investigation begins to take up too much time I will back the changes 
>> out.
>>
>> Best regards,
>> George
>>
>>
>> Stepan Mishura wrote:
>>> The same for me + DatagramChannelTest
>>>
>>> Thanks,
>>> Stepan.
>>>
>>> On 4/11/06, Mark Hindess wrote:
>>>
>>>> No. These:
>>>>
>>>> F
>>>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoder

>>>>
>>>> E
>>>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder01

>>>>
>>>> E
>>>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder02

>>>>
>>>> F org.apache.harmony.security.asn1.der.DerUTCTimeEDTest.testMt
>>>> E
>>>> org.apache.harmony.security.x509.PrivateKeyUsagePeriodTest.testEncodeDecode

>>>>
>>>> F java.security.cert.X509CertSelectorTest.testSetPrivateKeyValid
>>>> F java.security.cert.X509CertSelectorTest.testMatch
>>>> F java.security.cert.X509CertSelectorTest.testClone
>>>> F tests.api.java.sql.DateTest.testSetTimelong
>>>> F tests.api.java.sql.DateTest.testToString
>>>> F tests.api.java.sql.DateTest.testValueOf
>>>> F tests.api.java.sql.TimestampTest.testSetNanosint
>>>> F tests.api.java.sql.TimestampTest.testToString
>>>> F tests.api.java.util.DateTest.test_toGMTString
>>>> F tests.api.java.util.DateTest.test_toString
>>>>
>>>> Regards,
>>>> Mark.
>>>>
>>>> On 4/11/06, Stepan Mishura <stepan.mishura@gmail.com> wrote:
>>>>> On 4/11/06, Mark Hindess wrote:
>>>>>> Yes. I was using the failureproperty mechanism. Trying to get this
>>>>>> property propogated back to the top level ant file was what I was
>>>>>> having trouble with.
>>>>>>
>>>>>> Using a file as you suggest might help. I'll give that a try
>>>> shortly...
>>>>>> Incidentally, I'm seeing 12 failures and 3 errors on r393111.
>>>>> I guess that you have 9 tests from DatagramChannelTest passed. And 
>>>>> 12 +
>>>> 3 =
>>>>> 15 :-)
>>>>>
>>>>> (And
>>>>>> there are typos - "mathc" should be "match" - in the failure 
>>>>>> messages
>>>>>> for java.security.cert testMatch and testClone.)
>>>>> I've fixed typo.
>>>>>
>>>>> Thanks,
>>>>> Stepan.
>>>>>
>>>>> Regards,
>>>>>> Mark.
>>>>>>
>>>>>> On 4/11/06, Stepan Mishura <stepan.mishura@gmail.com> wrote:
>>>>>>> On 4/11/06, Mark Hindess wrote:
>>>>>>>
>>>>>>>> Stepan,
>>>>>>>>
>>>>>>>> I have another build running (but without notifications going
to
>>>> the
>>>>>>>> list) that runs:
>>>>>>>>
>>>>>>>> 1) build (with reference jdk)
>>>>>>>> 2) build with what we created with 1)
>>>>>>>> 3) test
>>>>>>>> 4) create classlists and compare with class load data for
>>>> applications
>>>>>>>> (tomcat, geronimo, continuum, etc.)
>>>>>>>> 5) generate JAPI results
>>>>>>>>
>>>>>>>> I'd like to fail this build at step 3, but I can't see an
easy way
>>>> to
>>>>>>>> get 'ant -f make/build.xml test' to run all tests and then
fail if
>>>> any
>>>>>>>> of the module test sub-targets had test failures. I could
parse
>>>> the
>>>>>>>> output I suppose, but I'd rather get ant to propagate the
junit
>>>> tasks
>>>>>>>> failure property back up to the top level. I've tried a couple
of
>>>>>>>> things and none seem to work. Any suggestions welcome.
>>>>>>> Mark, did you try failureproperty parameter for junit task?
>>>>>>> We may add it to ant sub-targets to raise a flag, for example,
>>>> create
>>>>>> file "
>>>>>>> TESTS.FAILED" in the root, when tests for some module fail. And
in
>>>> the
>>>>>> end
>>>>>>> of tests suite run check whether this file exists on not.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Stepan.
>>>>>>>
>>>>>>>
>>>>>>> Regards,
>>>>>>>> Mark.
>>>>>>>>
>>>>>>>> On 4/11/06, Stepan Mishura <stepan.mishura@gmail.com>
wrote:
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> I've checked out at morning last updates, built the code
base
>>>> and
>>>>>> run
>>>>>>>> the
>>>>>>>>> tests …and there are 24 tests failures!
>>>>>>>>>
>>>>>>>>> There are 9 tests failures in
>>>>>>>>> org.apache.harmony.tests.java.nio.channels.DatagramChannelTest
–
>>>> I
>>>>>> saw
>>>>>>>> these
>>>>>>>>> failures before from time to time. It seems that tests
depend on
>>>>>> some
>>>>>>>> race
>>>>>>>>> conditions because they may pass if I rerun them. Paulex,
are
>>>> these
>>>>>>>> tests
>>>>>>>>> passing for you?
>>>>>>>>>
>>>>>>>>> And there are new 15 test failures. So now if I'll make
a code
>>>>>> update
>>>>>>>> or a
>>>>>>>>> bug fix how I can be 100% sure that I don't do any regression?
>>>>>>>>>
>>>>>>>>> Currently if a commit breaks the Harmony classlib build
a
>>>>>> notification
>>>>>>>> with
>>>>>>>>> subject: "[continuum] BUILD FAILURE: Classlib/linux.ia32"
will
>>>> be
>>>>>> send
>>>>>>>> to
>>>>>>>>> harmony-commits mailing list. Is it possible to have
the same
>>>> for
>>>>>> tests?
>>>>>>>> I
>>>>>>>>> mean that after completing automatic build the existing
classlib
>>>>>> tests
>>>>>>>> suite
>>>>>>>>> should be run. If there are failing tests then a report
>>>> notification
>>>>>>>> with
>>>>>>>>> corresponding subject will be send.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Stepan Mishura
>>>>>>>>> Intel Middleware Products Division
>>>>>>>> -- 
>>>>>>>> Mark Hindess <mark.hindess@googlemail.com>
>>>>>>>> IBM Java Technology Centre, UK.
>>>>>>>>
>>>>>>>>
>>>> ---------------------------------------------------------------------
>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>>> To unsubscribe, e-mail:
>>>> harmony-dev-unsubscribe@incubator.apache.org
>>>>>>>> For additional commands, e-mail:
>>>> harmony-dev-help@incubator.apache.org
>>>>>>> -- 
>>>>>>>
>>>> ---------------------------------------------------------------------
>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>> To unsubscribe, e-mail: 
>>>>>>> harmony-dev-unsubscribe@incubator.apache.org
>>>>>>> For additional commands, e-mail:
>>>> harmony-dev-help@incubator.apache.org
>>>>>>> Thanks,
>>>>>>> Stepan Mishura
>>>>>>> Intel Middleware Products Division
>>>>>>>
>>>>>>>
>>>>>> -- 
>>>>>> Mark Hindess <mark.hindess@googlemail.com>
>>>>>> IBM Java Technology Centre, UK.
>>>>>>
>>>>>> ---------------------------------------------------------------------

>>>>>>
>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>>>> For additional commands, e-mail: 
>>>>>> harmony-dev-help@incubator.apache.org
>>>>>>
>>>>>>
>>>>> -- 
>>>>> ---------------------------------------------------------------------
>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>>> For additional commands, e-mail: 
>>>>> harmony-dev-help@incubator.apache.org
>>>>>
>>>>> Thanks,
>>>>> Stepan Mishura
>>>>> Intel Middleware Products Division
>>>>>
>>>>>
>>>> -- 
>>>> Mark Hindess <mark.hindess@googlemail.com>
>>>> IBM Java Technology Centre, UK.
>>>>
>>>> ---------------------------------------------------------------------
>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>>
>>>>
>>>
>>>
>>> -- 
>>> ---------------------------------------------------------------------
>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>
>>> Thanks,
>>> Stepan Mishura
>>> Intel Middleware Products Division
>>>
>>
>>
>> ---------------------------------------------------------------------
>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>


---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org


Mime
View raw message