harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From George Harley <george.c.har...@googlemail.com>
Subject Re: classlib test suite status emails?
Date Tue, 11 Apr 2006 12:45:16 GMT
Geir Magnusson Jr wrote:
> I forgot the smiley.

:-)

>
> I don't think this problem is so odd. Do you really think that 
> side-effects like this will be that rare?

Seriously, I don't really know. The case we have been discussing today 
was the first concrete example that I have encountered and I find the 
prospect of other occurrences sufficiently worrying that I'm all in 
favour of adding in some more Ant test targets along the lines of "test 
this module's dependencies". As pointed out by others, it's pretty 
daunting to have to run every test prior to a check-in and that is only 
bound to worsen as we grow the code base.

That's why I think that Stepan's idea of just running the dependencies' 
tests is a good one. It capitalizes on the componentized architecture of 
the class libraries.

Best regards,
George

>
> geir
>
> Geir Magnusson Jr wrote:
>> So the answer is "no"?
>>
>> George Harley wrote:
>>> Geir Magnusson Jr wrote:
>>>> Just curious (and this isn't a criticism - I'm just as guilty of 
>>>> not doing this)...
>>>>
>>>> Don't you run the tests before committing?
>>>
>>> Hi Geir,
>>>
>>> Depends what do you mean by "the tests".
>>>
>>> The change was completely encapsulated in the text component. I ran 
>>> the text tests. I checked that the new tests introduced in the issue 
>>> patch gave the same results when run against a 5.0 RI. All was fine.
>>>
>>> The automated build and test server (which runs all the tests in all 
>>> the modules) found the problem. That is it's job and it did it well. 
>>> The problem was found relatively quickly and has now been remedied. 
>>> The turnaround time would have been quicker if the automated build 
>>> and test server issued a failure notice on failing tests but that is 
>>> now in the process of getting sorted out.
>>>
>>> IMHO this problem was a bit of an odd problem as the failures did 
>>> not arise in the patched module's test suite but in other modules. 
>>> That is a good indication that the test suite for text is in need of 
>>> some scrutiny. That is a useful thing to be made aware of.
>>>
>>>
>>> Best regards,
>>> George
>>>
>>>>
>>>> geir
>>>>
>>>> George Harley wrote:
>>>>> Hi,
>>>>>
>>>>> It *seems* like things started failing after I committed the 
>>>>> changes for HARMONY-205 last night. I'm looking into this now. If 
>>>>> the investigation begins to take up too much time I will back the 
>>>>> changes out.
>>>>>
>>>>> Best regards,
>>>>> George
>>>>>
>>>>>
>>>>> Stepan Mishura wrote:
>>>>>> The same for me + DatagramChannelTest
>>>>>>
>>>>>> Thanks,
>>>>>> Stepan.
>>>>>>
>>>>>> On 4/11/06, Mark Hindess wrote:
>>>>>>
>>>>>>> No. These:
>>>>>>>
>>>>>>> F
>>>>>>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoder

>>>>>>>
>>>>>>> E
>>>>>>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder01

>>>>>>>
>>>>>>> E
>>>>>>> org.apache.harmony.security.asn1.der.DerGeneralizedTimeEDTest.testGeneralizedEncoderDecoder02

>>>>>>>
>>>>>>> F org.apache.harmony.security.asn1.der.DerUTCTimeEDTest.testMt
>>>>>>> E
>>>>>>> org.apache.harmony.security.x509.PrivateKeyUsagePeriodTest.testEncodeDecode

>>>>>>>
>>>>>>> F java.security.cert.X509CertSelectorTest.testSetPrivateKeyValid
>>>>>>> F java.security.cert.X509CertSelectorTest.testMatch
>>>>>>> F java.security.cert.X509CertSelectorTest.testClone
>>>>>>> F tests.api.java.sql.DateTest.testSetTimelong
>>>>>>> F tests.api.java.sql.DateTest.testToString
>>>>>>> F tests.api.java.sql.DateTest.testValueOf
>>>>>>> F tests.api.java.sql.TimestampTest.testSetNanosint
>>>>>>> F tests.api.java.sql.TimestampTest.testToString
>>>>>>> F tests.api.java.util.DateTest.test_toGMTString
>>>>>>> F tests.api.java.util.DateTest.test_toString
>>>>>>>
>>>>>>> Regards,
>>>>>>> Mark.
>>>>>>>
>>>>>>> On 4/11/06, Stepan Mishura <stepan.mishura@gmail.com> wrote:
>>>>>>>> On 4/11/06, Mark Hindess wrote:
>>>>>>>>> Yes. I was using the failureproperty mechanism. Trying
to get 
>>>>>>>>> this
>>>>>>>>> property propogated back to the top level ant file was
what I was
>>>>>>>>> having trouble with.
>>>>>>>>>
>>>>>>>>> Using a file as you suggest might help. I'll give that
a try
>>>>>>> shortly...
>>>>>>>>> Incidentally, I'm seeing 12 failures and 3 errors on
r393111.
>>>>>>>> I guess that you have 9 tests from DatagramChannelTest passed.

>>>>>>>> And 12 +
>>>>>>> 3 =
>>>>>>>> 15 :-)
>>>>>>>>
>>>>>>>> (And
>>>>>>>>> there are typos - "mathc" should be "match" - in the
failure 
>>>>>>>>> messages
>>>>>>>>> for java.security.cert testMatch and testClone.)
>>>>>>>> I've fixed typo.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Stepan.
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>>> Mark.
>>>>>>>>>
>>>>>>>>> On 4/11/06, Stepan Mishura <stepan.mishura@gmail.com>
wrote:
>>>>>>>>>> On 4/11/06, Mark Hindess wrote:
>>>>>>>>>>
>>>>>>>>>>> Stepan,
>>>>>>>>>>>
>>>>>>>>>>> I have another build running (but without notifications

>>>>>>>>>>> going to
>>>>>>> the
>>>>>>>>>>> list) that runs:
>>>>>>>>>>>
>>>>>>>>>>> 1) build (with reference jdk)
>>>>>>>>>>> 2) build with what we created with 1)
>>>>>>>>>>> 3) test
>>>>>>>>>>> 4) create classlists and compare with class load
data for
>>>>>>> applications
>>>>>>>>>>> (tomcat, geronimo, continuum, etc.)
>>>>>>>>>>> 5) generate JAPI results
>>>>>>>>>>>
>>>>>>>>>>> I'd like to fail this build at step 3, but I
can't see an 
>>>>>>>>>>> easy way
>>>>>>> to
>>>>>>>>>>> get 'ant -f make/build.xml test' to run all tests
and then 
>>>>>>>>>>> fail if
>>>>>>> any
>>>>>>>>>>> of the module test sub-targets had test failures.
I could parse
>>>>>>> the
>>>>>>>>>>> output I suppose, but I'd rather get ant to propagate
the junit
>>>>>>> tasks
>>>>>>>>>>> failure property back up to the top level. I've
tried a 
>>>>>>>>>>> couple of
>>>>>>>>>>> things and none seem to work. Any suggestions
welcome.
>>>>>>>>>> Mark, did you try failureproperty parameter for junit
task?
>>>>>>>>>> We may add it to ant sub-targets to raise a flag,
for example,
>>>>>>> create
>>>>>>>>> file "
>>>>>>>>>> TESTS.FAILED" in the root, when tests for some module
fail. 
>>>>>>>>>> And in
>>>>>>> the
>>>>>>>>> end
>>>>>>>>>> of tests suite run check whether this file exists
on not.
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Stepan.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Regards,
>>>>>>>>>>> Mark.
>>>>>>>>>>>
>>>>>>>>>>> On 4/11/06, Stepan Mishura <stepan.mishura@gmail.com>
wrote:
>>>>>>>>>>>> Hi,
>>>>>>>>>>>>
>>>>>>>>>>>> I've checked out at morning last updates,
built the code base
>>>>>>> and
>>>>>>>>> run
>>>>>>>>>>> the
>>>>>>>>>>>> tests …and there are 24 tests failures!
>>>>>>>>>>>>
>>>>>>>>>>>> There are 9 tests failures in
>>>>>>>>>>>> org.apache.harmony.tests.java.nio.channels.DatagramChannelTest

>>>>>>>>>>>> –
>>>>>>> I
>>>>>>>>> saw
>>>>>>>>>>> these
>>>>>>>>>>>> failures before from time to time. It seems
that tests 
>>>>>>>>>>>> depend on
>>>>>>>>> some
>>>>>>>>>>> race
>>>>>>>>>>>> conditions because they may pass if I rerun
them. Paulex, are
>>>>>>> these
>>>>>>>>>>> tests
>>>>>>>>>>>> passing for you?
>>>>>>>>>>>>
>>>>>>>>>>>> And there are new 15 test failures. So now
if I'll make a code
>>>>>>>>> update
>>>>>>>>>>> or a
>>>>>>>>>>>> bug fix how I can be 100% sure that I don't
do any regression?
>>>>>>>>>>>>
>>>>>>>>>>>> Currently if a commit breaks the Harmony
classlib build a
>>>>>>>>> notification
>>>>>>>>>>> with
>>>>>>>>>>>> subject: "[continuum] BUILD FAILURE: Classlib/linux.ia32"
will
>>>>>>> be
>>>>>>>>> send
>>>>>>>>>>> to
>>>>>>>>>>>> harmony-commits mailing list. Is it possible
to have the same
>>>>>>> for
>>>>>>>>> tests?
>>>>>>>>>>> I
>>>>>>>>>>>> mean that after completing automatic build
the existing 
>>>>>>>>>>>> classlib
>>>>>>>>> tests
>>>>>>>>>>> suite
>>>>>>>>>>>> should be run. If there are failing tests
then a report
>>>>>>> notification
>>>>>>>>>>> with
>>>>>>>>>>>> corresponding subject will be send.
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> Stepan Mishura
>>>>>>>>>>>> Intel Middleware Products Division
>>>>>>>>>>> -- 
>>>>>>>>>>> Mark Hindess <mark.hindess@googlemail.com>
>>>>>>>>>>> IBM Java Technology Centre, UK.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>> ---------------------------------------------------------------------

>>>>>>>
>>>>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>>>>>> To unsubscribe, e-mail:
>>>>>>> harmony-dev-unsubscribe@incubator.apache.org
>>>>>>>>>>> For additional commands, e-mail:
>>>>>>> harmony-dev-help@incubator.apache.org
>>>>>>>>>> -- 
>>>>>>>>>>
>>>>>>> ---------------------------------------------------------------------

>>>>>>>
>>>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>>>>> To unsubscribe, e-mail: 
>>>>>>>>>> harmony-dev-unsubscribe@incubator.apache.org
>>>>>>>>>> For additional commands, e-mail:
>>>>>>> harmony-dev-help@incubator.apache.org
>>>>>>>>>> Thanks,
>>>>>>>>>> Stepan Mishura
>>>>>>>>>> Intel Middleware Products Division
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> -- 
>>>>>>>>> Mark Hindess <mark.hindess@googlemail.com>
>>>>>>>>> IBM Java Technology Centre, UK.
>>>>>>>>>
>>>>>>>>> ---------------------------------------------------------------------

>>>>>>>>>
>>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>>>> To unsubscribe, e-mail: 
>>>>>>>>> harmony-dev-unsubscribe@incubator.apache.org
>>>>>>>>> For additional commands, e-mail: 
>>>>>>>>> harmony-dev-help@incubator.apache.org
>>>>>>>>>
>>>>>>>>>
>>>>>>>> -- 
>>>>>>>> ---------------------------------------------------------------------

>>>>>>>>
>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>>> To unsubscribe, e-mail: 
>>>>>>>> harmony-dev-unsubscribe@incubator.apache.org
>>>>>>>> For additional commands, e-mail: 
>>>>>>>> harmony-dev-help@incubator.apache.org
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Stepan Mishura
>>>>>>>> Intel Middleware Products Division
>>>>>>>>
>>>>>>>>
>>>>>>> -- 
>>>>>>> Mark Hindess <mark.hindess@googlemail.com>
>>>>>>> IBM Java Technology Centre, UK.
>>>>>>>
>>>>>>> ---------------------------------------------------------------------

>>>>>>>
>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>> To unsubscribe, e-mail: 
>>>>>>> harmony-dev-unsubscribe@incubator.apache.org
>>>>>>> For additional commands, e-mail: 
>>>>>>> harmony-dev-help@incubator.apache.org
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> -- 
>>>>>> ---------------------------------------------------------------------

>>>>>>
>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>>>> For additional commands, e-mail: 
>>>>>> harmony-dev-help@incubator.apache.org
>>>>>>
>>>>>> Thanks,
>>>>>> Stepan Mishura
>>>>>> Intel Middleware Products Division
>>>>>>
>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>>> For additional commands, e-mail: 
>>>>> harmony-dev-help@incubator.apache.org
>>>>>
>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>>
>>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>


---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org


Mime
View raw message