geronimo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alex Blewitt <>
Subject Re: [Testing] Testing
Date Tue, 12 Aug 2003 08:24:44 GMT

On Tuesday, Aug 12, 2003, at 07:37 Europe/London, Bruce Snyder wrote:

> 2) Another issue is how to label the spec compliance tests in order to
> parse them later for reporting, etc.
> For this issue, not only should each spec compliance test be fully
> documented via Javadoc, but we need to devise some meta tags for 
> labeling
> each test case for doclet parsing. I suggest that we use the following:
> @spec
>     name="J2EE"
>     version="1.4"
>     chapter-name="Transaction Management"
>     chapter-number="4"
>     chapter-section-name="Transactional JDBC Technology Support"
>     chapter-section-number="4.2.7"

Can you tie spec tests down to a specific page/chapter, though? I'm 
sure that the relationship isn't 1-1. For example, a spec requirement 
may come out of several places in the document; at the very least, on 
different pages. Further, there may be some spec tests which cover many 
different types of element, or that go into more detail than the spec 
document allows (such as CMP mapping, which isn't covered by the spec 
at all).

I'm also pretty sure that the -name and -number are storing duplicate 
information; you'd really only need the -number ones, and provide a 
lookup file elsewhere. In fact, given that the section stores 
chapter-and-section number, you'd really only need that :-)

> Feedback? Better ideas?

Another approach would be to plan the tests on the Wiki first, and then 
write the tests themselves, rather than the spec tests being self 
documenting. The former approach will let you build a complete list of 
tests first; however, if you rely on the individual tests saying that 
'section 4.2.7' is tested, then that can lull you into a false sense of 
security; yes, /one/ of the tests in 4.2.7 may be covered, but that 
doesn't mean that /all/ of them will be.


View raw message