harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paulex Yang <paulex.y...@gmail.com>
Subject Re: Test suite layout
Date Thu, 12 Jan 2006 16:56:36 GMT
Geir Magnusson Jr wrote:

>
>
> Tim Ellison wrote:
>
>> What we've tended to do, internally, is to name the testcases after the
>> type they are testing, so all unit tests for java.io.File are put into a
>> tests package ending with java.io in a type called FileTest that extends
>> the junit.framework.TestCase.
>
>
> Yes - that's the canonical way.  Put it the the same package as what 
> you are testing, and then use a common name pattern to make it easy to 
> configure ant or maven with a wildcard.
>
>   */*/*Test
>
>
>>
>> We would have written it as java.io.tests, but the java.<whatever>
>> namespace is reserved, so the formula is simply
>>
>> <package>.<type>  ->   org.apache.harmony.tests.<package>.<type>Test
>
>
> Ug - then you have the problem of not being in the namespace as what 
> you are testing.
>
> THat's why people use parallel trees - so your test code is physically 
> separate but you have freedom of package access.
>
>>
>> This makes it clear what is being tested, and where to add new tests 
>> etc.
>
>
> So would
>
>   test/org/apache/harmony/io/TestFoo.java
>
> (to test something in org.apache.harmony.io, and arguable to test the 
> Foo.java class in there.  (or TestFoo.java - it's early - no coffee yet)
>
> Simiarly
>
>   test/java/util/TestMap.java
>
>>
>> Then within the test class itself the methods are named after the method
>> under test, with a familar JNI-style encoding,  so we have things like:
>>
>> org.apache.harmony.tests.java.io.FileTest contains
>>     public void test_ConstructorLjava_io_FileLjava_lang_String() {
>>     ...
>>     }
>>
>> and
>>
>> org.apache.harmony.tests.java.lang.FloatTest contains
>>     public void test_compareToLjava_lang_Float() {
>>     ...
>>     }
>
>
> ...or whatever the convention is for JUnit.  I think that's one of the 
> nice things about TestNG, is that it's annotated, so you have the 
> freedom there.

Agree with you that annotation is great, and I just wish JUnit 4 can be 
released soon
(check out an intro here
http://www-128.ibm.com/developerworks/java/library/j-junit4.html)

But even if we have more freedom, I think some convention is still 
*precious*, this name contract between test and method is one example. At 
least in Eclipse, I can easily find the test case for some certain 
method simply by ctrl+o and key in "test_"+method name, that's natural 
and easy, and if tests failed, I can see which method is wrong in just 
one glance, so personally I'm willing to pay the price to convention 8-) 

>>
>>
>> If the test is added due to a regression, then it is put into the right
>> place in the test suite, and flagged with a comment (i.e. a reference to
>> the Harmony JIRA number).
>
>
> Yes - and I'd even advocate a parallel directory there too so that 
> it's clear that the regressions are different, but whatever.  The only 
> snag there is name collision with the classes.
>
> I think that a simple comment is enough.  If we want to get cute, 
> maybe a javadoc tag so we can manage mechanically in the future.
>
> geir
>
>>
>> Regards,
>> Tim
>>
>>
>> George Harley1 wrote:
>>
>>> Hi,
>>>
>>>
>>>> I think that regression tests should be marked in some way.
>>>
>>>
>>>
>>> Agreed.  But can we please *resist* the temptation to do this by 
>>> incorporating JIRA issue numbers into test case names (e.g. calling 
>>> unit test methods test_26() or test_JIRA_26()). I've seen this kind 
>>> of approach adopted in a couple of projects and, in my experience, 
>>> it often leads to the scattering of duplicated test code around the 
>>> test harness.
>>>
>>> Better, methinks, to either create a new test method with a 
>>> meaningful name or else augment an existing method - whatever makes 
>>> more sense for the particular issue. Then marking certain code as 
>>> being for regression test purposes could be done in comments that 
>>> include the URL of the JIRA issue. Perhaps an agreed tag like "JIRA" 
>>> or "BUG" etc could be used as an eye-catcher as well ?
>>> e.g.
>>> // BUG http://issues.apache.org/jira/browse/HARMONY-26
>>>
>>>
>>> My 2 Euro Cents.
>>> Best regards, George
>>> ________________________________________
>>> George C. Harley
>>>
>>>
>>>
>>>
>>> "Mishura, Stepan M" <stepan.m.mishura@intel.com> 12/01/2006 04:56
>>> Please respond to
>>> harmony-dev@incubator.apache.org
>>>
>>>
>>> To
>>> <harmony-dev@incubator.apache.org>
>>> cc
>>>
>>> Subject
>>> RE: regression test suite
>>>
>>>
>>>
>>>
>>>
>>>
>>> Hello,
>>> Tim Ellison wrote:
>>> [snip]
>>>
>>>
>>>> What is the useful distinction for regression tests being kept
>>>
>>>
>>> separate?
>>>
>>>
>>>> I can see that you may distinguish unit and 'system-level' tests just
>>>> because of the difference in frameworks etc. required, but why do I
>>>
>>>
>>> care
>>>
>>>
>>>> if the test was written due to a JIRA issue or test-based development
>>>
>>>
>>> or
>>>
>>>
>>>> someone who get's kicks out of writing tests to break the code?
>>>>
>>>
>>>
>>> I agree that separating regression tests doesn't make sense.
>>> However I think that regression tests should be marked in some way.
>>> This will signal a developer that a test was created to track already
>>> known issue. IMHO, a regression test should point out to a bug report
>>> and a bug report (after resolving a bug) should contain a reference to
>>> corresponding regression test in repository.
>>>
>>> Thanks,
>>> Stepan Mishura
>>> Intel Middleware Products Division
>>>
>>>
>>>
>>
>>
>


-- 
Paulex Yang
China Software Development Lab
IBM


Mime
View raw message