harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From George Harley <george.c.har...@googlemail.com>
Subject Re: [classlib][testing] excluding the failed tests
Date Mon, 10 Jul 2006 12:06:16 GMT
Alexei Zakharov wrote:
> Hi,
>
>> If there are really useful tests that are being unnecessarily excluded
>> by being in the same *Test class, then you may want to consider moving
>> the failing tests into SecureRandom3Test and excluding that -- but by
>> the sound of it all SecureRandom tests will be failing.
>
> I think it's a nice idea to do this at least for java.beans since
> there are hundreds of useful workable tests excluded. After quite a
> long time working with this module I have a strong wish to clean up
> the mess.
>
> But probably we should define some naming pattern for class to put
> excluded tests into. For example for XMLEncoderTest.java we can have
> XMLEncoderTest_Disabled.java or XMLEncoderTest_Failed.java. In this
> case we don't need to put extra "exclude" clause in the build.xml
> since such name doesn't match **/*Test.java pattern (current). Another
> variant is something like FAILED_XMLEncoderTest.java - matches the
> pattern and needs the clause. Thoughts?

Hi Alexei,

Have you seen the discussion thread related to configuring our tests 
using suites [1] ? If not, then it seems to me that there is potential 
there for a simpler/quicker way of excluding or including tests without 
recourse to creating new files or renaming existing ones. What do you 
think ?

Best regards,
George

[1] 
http://mail-archives.apache.org/mod_mbox/incubator-harmony-dev/200607.mbox/%3c44ABB451.30806@googlemail.com%3e


>
>
> 2006/7/6, Tim Ellison <t.p.ellison@gmail.com>:
>> Vladimir Ivanov wrote:
>> > More details: it is
>> > org/apache/harmony/security/tests/java/security/SecureRandom2Test.java
>> > test.
>> > At present time it has 2 failing tests with messages about SHA1PRNG
>> > algorithm (no support for SHA1PRNG provider).
>> > Looks like it is valid tests for non implemented functionality, 
>> but, I'm
>> > not
>> > sure what to do with such TestCase(s): comment these 2 tests or 
>> move them
>> > into separate TestCase.
>> > Ideas?
>>
>> I'd prefer that we only use one mechanism for excluding tests, and today
>> that is the excludes clause in the ant script.  So I suggest that you do
>> option (4) below.
>>
>> If there are really useful tests that are being unnecessarily excluded
>> by being in the same *Test class, then you may want to consider moving
>> the failing tests into SecureRandom3Test and excluding that -- but by
>> the sound of it all SecureRandom tests will be failing.
>>
>> > By the way, probably, it worth reviewing *all* excluded TestCases and:
>> > 1.      Unexclude if all tests pass.
>> > 2.      Report bug and provide patch for test to make it passing if it
>> > failed due to bug in test.
>> > 3.      Report bug (and provide patch) for implementation to make 
>> tests
>> > passing, if it was/is bug in implementation and no such issue in JIRA.
>> > 4.      Specify reasons for excluding TestCases in exclude list to 
>> make
>> > further clean-up process easier.
>> > 5.      Review results of this exclude list clean-up activity and then
>> > decide what to do with the rest failing tests.
>> >
>> > I can do it starting next week. Do you think it worth doing?
>> > Thanks, Vladimir
>>
>> Sounds great, thanks Vladimir.
>>
>> Regards,
>> Tim
>
>


---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org


Mime
View raw message