Return-Path: 
 <harmony-dev-return-2993-apmail-incubator-harmony-dev-archive=incubator.apache.org@incubator.apache.org>
Delivered-To: apmail-incubator-harmony-dev-archive@www.apache.org
Received: (qmail 4501 invoked from network); 12 Jan 2006 11:27:52 -0000
Received: from hermes.apache.org (HELO mail.apache.org) (209.237.227.199)
  by minotaur.apache.org with SMTP; 12 Jan 2006 11:27:52 -0000
Received: (qmail 82259 invoked by uid 500); 12 Jan 2006 11:27:48 -0000
Delivered-To: apmail-incubator-harmony-dev-archive@incubator.apache.org
Received: (qmail 82010 invoked by uid 500); 12 Jan 2006 11:27:46 -0000
Mailing-List: contact harmony-dev-help@incubator.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:harmony-dev-help@incubator.apache.org>
List-Unsubscribe: <mailto:harmony-dev-unsubscribe@incubator.apache.org>
List-Post: <mailto:harmony-dev@incubator.apache.org>
List-Id: <harmony-dev.incubator.apache.org>
Reply-To: harmony-dev@incubator.apache.org
Delivered-To: mailing list harmony-dev@incubator.apache.org
Received: (qmail 81995 invoked by uid 99); 12 Jan 2006 11:27:46 -0000
Received: from asf.osuosl.org (HELO asf.osuosl.org) (140.211.166.49)
    by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 12 Jan 2006 03:27:46 -0800
X-ASF-Spam-Status: No, hits=2.6 required=10.0
	tests=ADVANCE_FEE_1,RCVD_IN_SORBS_WEB,SPF_NEUTRAL
X-Spam-Check-By: apache.org
Received-SPF: neutral (asf.osuosl.org: 217.158.94.220 is neither permitted nor
 denied by domain of t.p.ellison@gmail.com)
Received: from [217.158.94.220] (HELO cirrus.purplecloud.com) (217.158.94.220)
    by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 12 Jan 2006 03:27:45 -0800
Received: (qmail 96387 invoked from network); 12 Jan 2006 11:27:23 +0000
Received: from blueice2n1.uk.ibm.com (HELO ?9.20.183.163?) (195.212.29.75)
  by smtp.purplecloud.net with (DHE-RSA-AES256-SHA encrypted) SMTP;
 12 Jan 2006 11:27:22 +0000
Message-ID: <43C63D19.60603@gmail.com>
Date: Thu, 12 Jan 2006 11:27:21 +0000
From: Tim Ellison <t.p.ellison@gmail.com>
User-Agent: Mozilla Thunderbird 1.0.7 (Windows/20050923)
X-Accept-Language: en-us, en
MIME-Version: 1.0
To: harmony-dev@incubator.apache.org
Subject: Test suite layout (was: Re: regression test suite)
References: 
 <OFC4ACA33D.81511539-ON802570F4.0033370C-802570F4.0035E4EA@uk.ibm.com>
In-Reply-To: 
 <OFC4ACA33D.81511539-ON802570F4.0033370C-802570F4.0035E4EA@uk.ibm.com>
X-Enigmail-Version: 0.93.0.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
X-Virus-Checked: Checked by ClamAV on apache.org
X-Spam-Rating: minotaur.apache.org 1.6.2 0/1000/N

What we've tended to do, internally, is to name the testcases after the
type they are testing, so all unit tests for java.io.File are put into a
tests package ending with java.io in a type called FileTest that extends
the junit.framework.TestCase.

We would have written it as java.io.tests, but the java.<whatever>
namespace is reserved, so the formula is simply

<package>.<type>  ->   org.apache.harmony.tests.<package>.<type>Test

This makes it clear what is being tested, and where to add new tests etc.

Then within the test class itself the methods are named after the method
under test, with a familar JNI-style encoding,  so we have things like:

org.apache.harmony.tests.java.io.FileTest contains
	public void test_ConstructorLjava_io_FileLjava_lang_String() {
	...
	}

and

org.apache.harmony.tests.java.lang.FloatTest contains
	public void test_compareToLjava_lang_Float() {
	...
	}


If the test is added due to a regression, then it is put into the right
place in the test suite, and flagged with a comment (i.e. a reference to
the Harmony JIRA number).

Regards,
Tim


George Harley1 wrote:
> Hi, 
> 
> 
>>I think that regression tests should be marked in some way.
> 
> 
> Agreed.  But can we please *resist* the temptation to do this by 
> incorporating JIRA issue numbers into test case names (e.g. calling unit 
> test methods test_26() or test_JIRA_26()). I've seen this kind of approach 
> adopted in a couple of projects and, in my experience, it often leads to 
> the scattering of duplicated test code around the test harness.
> 
> Better, methinks, to either create a new test method with a meaningful 
> name or else augment an existing method - whatever makes more sense for 
> the particular issue. Then marking certain code as being for regression 
> test purposes could be done in comments that include the URL of the JIRA 
> issue. Perhaps an agreed tag like "JIRA" or "BUG" etc could be used as an 
> eye-catcher as well ?
> e.g. 
> 
> // BUG http://issues.apache.org/jira/browse/HARMONY-26
> 
> 
> My 2 Euro Cents. 
> 
> Best regards, 
> George
> ________________________________________
> George C. Harley
> 
> 
> 
> 
> "Mishura, Stepan M" <stepan.m.mishura@intel.com> 
> 12/01/2006 04:56
> Please respond to
> harmony-dev@incubator.apache.org
> 
> 
> To
> <harmony-dev@incubator.apache.org>
> cc
> 
> Subject
> RE: regression test suite
> 
> 
> 
> 
> 
> 
> Hello, 
> 
> Tim Ellison wrote: 
> 
> [snip]
> 
>>What is the useful distinction for regression tests being kept
> 
> separate?
> 
>>I can see that you may distinguish unit and 'system-level' tests just
>>because of the difference in frameworks etc. required, but why do I
> 
> care
> 
>>if the test was written due to a JIRA issue or test-based development
> 
> or
> 
>>someone who get's kicks out of writing tests to break the code?
>>
> 
> 
> I agree that separating regression tests doesn't make sense.
> However I think that regression tests should be marked in some way.
> This will signal a developer that a test was created to track already
> known issue. IMHO, a regression test should point out to a bug report
> and a bug report (after resolving a bug) should contain a reference to
> corresponding regression test in repository.
> 
> Thanks,
> Stepan Mishura
> Intel Middleware Products Division
> 
> 
> 

-- 

Tim Ellison (t.p.ellison@gmail.com)
IBM Java technology centre, UK.