geronimo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tim Anderson" <...@netspace.net.au>
Subject RE: [Testing] Testing
Date Tue, 12 Aug 2003 23:07:01 GMT
See inline

> From: Bruce Snyder [mailto:ferret@frii.com]
>
> This one time, at band camp, Tim Anderson said:
>
> TA>The JMS CTS (http://jmscts.sf.net) does something similar,
> TA>except that rather than using section numbers, it associates
> TA>an identifier with each requirement.
> TA>
> TA>A requirement can then be associated with multiple sections of
> TA>the JMS spec, useful when a requirement is covered in multiple
> TA>places.
> TA>
> TA>It also saves on verbosity, which helps if more than one test
> TA>case tests a requirement.
> TA>
> TA>All requirements are maintained in a .xml file e.g.:
> TA>
> TA>   ...
> TA>  <reference referenceId="section.6.11.1">
> TA>    <section name="6.11.1" title="Durable Topic Subscriber" />
> TA>  </reference>
> TA>   ...
> TA>
> TA>  <requirement requirementId="subscriber.durable.unique">
> TA>    <description>
> TA>      Only one session at a time can have a TopicSubscriber for a
> TA>      particular durable subscription.
> TA>    </description>
> TA>    <referenceId>section.6.11.1</referenceId>
> TA>    <referenceId>section.4.3.2</referenceId>
> TA>  </requirement>
> TA>
> TA>(from
> TA>http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/jmscts/jmscts/sr
> c/java/org/ex
> TA>olab/jmscts/requirements/requirements.xml)
> TA>
> TA>And referenced in code using the @jmscts.requirement tag e.g:
> TA>
> TA>    /**
> TA>     * Verifies that creating a duplicate durable subscriber
> in the same
> TA>     * session throws JMSException
> TA>     *
> TA>     * @jmscts.requirement subscriber.durable.unique
> TA>     * @throws Exception for any error
> TA>     */
> TA>    public void testDuplicateSubscriberPerSession() throws Exception {
>
> Actually, now that I see this, I do think that it flows better. I like the
> fact that the code is less littered. My previous objection was regarding
> the maintenane of two files. However, this was more in regards to creating
> the file ahead of time. If the requirements.xml was maintained as part
> of the development of the tests, this would certainly be manageable.
>
> Tim, how do you handle the previous concern raised by Alex where a single
> requirement spanning many pages is covered by many test cases?

In the CTS, multiple test cases can reference a single requirement.
For reporting, it registers a TestListener to track test the results
of test cases, and their corresponding requirements.

At the end of the test run it:
1.  generates an .xml file which records test results and the requirements
    covered
2.  uses XSLT to produce xdoc and HTML reports. These include the text
    from requirements.xml

> Also, how
> do you handle the difference between the Javadoc and the requirements.xml
> <description> element when generating a report? It seems that an HTML
> generated report should contain the documentation, but which one should
> it include if the two are different?
>
> Bruce

I don't. All text in the final report is extracted from requirements.xml.
Incorporating javadoc I put in the too hard to be bothered basket.
Where a requirement is specified in javadoc rather than the spec, I
generally just make the description 'Refer to javadoc', and the <reference>
gets added to the final report.


-Tim



Mime
View raw message