Mailing-List: contact harmony-dev-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: harmony-dev@incubator.apache.org
Received-SPF: pass (asf.osuosl.org: domain of george.c.harley@googlemail.com
 designates 64.233.182.188 as permitted sender)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws;
        s=beta; d=googlemail.com;
        h=received:message-id:date:from:reply-to:user-agent:mime-version:to:subject:references:in-reply-to:content-type:content-transfer-encoding;
        b=HWEGr6WnyFirw0+DmtmBSzr3aVYzJFNDRrbgazJJqoAgLIjjdB7Bm0146QeMfOa4DRKdgEOf0WVNc6edTncjojbXoQzgi8DnpqY9NGbFUbQnprf7XIJ9aFtyPOqZ9pK8ij2wdy4VktzC/XGApkB835rJVJ9GZtKmfFs8d2J1JiM=
Message-ID: <44B242B8.1020200@googlemail.com>
Date: Mon, 10 Jul 2006 13:06:16 +0100
From: George Harley <george.c.harley@googlemail.com>
Reply-To: harmony-dev@incubator.apache.org
User-Agent: Thunderbird 1.5.0.4 (Windows/20060516)
MIME-Version: 1.0
To: harmony-dev@incubator.apache.org
Subject: Re: [classlib][testing] excluding the failed tests
References: <7273946b0607042240h7a91bef6n45c2a4e03af6c250@mail.gmail.com>
	 <000d01c6a0a1$a5714300$0e01a8c0@LITTLEGUY>
	 <7273946b0607060252l54165ec4h99f56cc83dbc7526@mail.gmail.com>
	 <44AD4FBB.501@gmail.com>
 <2c9597b90607100435w4b245511xbc1deeaf67ec77d4@mail.gmail.com>
In-Reply-To: <2c9597b90607100435w4b245511xbc1deeaf67ec77d4@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Alexei Zakharov wrote:
> Hi,
>
>> If there are really useful tests that are being unnecessarily excluded
>> by being in the same *Test class, then you may want to consider moving
>> the failing tests into SecureRandom3Test and excluding that -- but by
>> the sound of it all SecureRandom tests will be failing.
>
> I think it's a nice idea to do this at least for java.beans since
> there are hundreds of useful workable tests excluded. After quite a
> long time working with this module I have a strong wish to clean up
> the mess.
>
> But probably we should define some naming pattern for class to put
> excluded tests into. For example for XMLEncoderTest.java we can have
> XMLEncoderTest_Disabled.java or XMLEncoderTest_Failed.java. In this
> case we don't need to put extra "exclude" clause in the build.xml
> since such name doesn't match **/*Test.java pattern (current). Another
> variant is something like FAILED_XMLEncoderTest.java - matches the
> pattern and needs the clause. Thoughts?

Hi Alexei,

Have you seen the discussion thread related to configuring our tests 
using suites [1] ? If not, then it seems to me that there is potential 
there for a simpler/quicker way of excluding or including tests without 
recourse to creating new files or renaming existing ones. What do you 
think ?

Best regards,
George

[1] 
http://mail-archives.apache.org/mod_mbox/incubator-harmony-dev/200607.mbox/%3c44ABB451.30806@googlemail.com%3e


>
>
> 2006/7/6, Tim Ellison <t.p.ellison@gmail.com>:
>> Vladimir Ivanov wrote:
>> > More details: it is
>> > org/apache/harmony/security/tests/java/security/SecureRandom2Test.java
>> > test.
>> > At present time it has 2 failing tests with messages about SHA1PRNG
>> > algorithm (no support for SHA1PRNG provider).
>> > Looks like it is valid tests for non implemented functionality, 
>> but, I'm
>> > not
>> > sure what to do with such TestCase(s): comment these 2 tests or 
>> move them
>> > into separate TestCase.
>> > Ideas?
>>
>> I'd prefer that we only use one mechanism for excluding tests, and today
>> that is the excludes clause in the ant script.  So I suggest that you do
>> option (4) below.
>>
>> If there are really useful tests that are being unnecessarily excluded
>> by being in the same *Test class, then you may want to consider moving
>> the failing tests into SecureRandom3Test and excluding that -- but by
>> the sound of it all SecureRandom tests will be failing.
>>
>> > By the way, probably, it worth reviewing *all* excluded TestCases and:
>> > 1.      Unexclude if all tests pass.
>> > 2.      Report bug and provide patch for test to make it passing if it
>> > failed due to bug in test.
>> > 3.      Report bug (and provide patch) for implementation to make 
>> tests
>> > passing, if it was/is bug in implementation and no such issue in JIRA.
>> > 4.      Specify reasons for excluding TestCases in exclude list to 
>> make
>> > further clean-up process easier.
>> > 5.      Review results of this exclude list clean-up activity and then
>> > decide what to do with the rest failing tests.
>> >
>> > I can do it starting next week. Do you think it worth doing?
>> > Thanks, Vladimir
>>
>> Sounds great, thanks Vladimir.
>>
>> Regards,
>> Tim
>
>


---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org