xml-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kelly Campbell <c...@channelpoint.com>
Subject RE: Test Infrastructure Project Proposal
Date Mon, 12 Feb 2001 21:55:04 GMT
Scott,

I think another requirement that you've covered in a roundabout way, but
should be added explicitely is the test suites should be easily runnable by
individual developers, and the full suite should be run prior to committing
code changes.

-Kelly

-----Original Message-----
From: Scott_Boag@lotus.com [mailto:Scott_Boag@lotus.com]
Sent: Monday, February 12, 2001 1:44 PM
To: general@xml.apache.org
Subject: Re: Test Infrastructure Project Proposal



Here's a start on Test Infrastructure requirements, rather unstructured,
and off the top of my head:

* It must be able to have complex traversals of input directories.
* It should support easy running of categories of tests.
* A mechanism must exist to hand in datafiles to a single test program.
(In concrete, we need to run Xalan over some 1500 tests pairs of XSLT and
XML).
* Must have pluggable result comparitors (for instance, to compare XML
disregarding attribute order, to compare HTML disregarding non-significant
whitespace, etc.)
* Tests should be easily run stand-alone from the main infrastructure, for
easy debugging and problem isolation.  (This means, I don't like having to
subclass my test classes)
* Per-platform exclusions should be able to be specified (for instance,
known platforms that do not support certain encodings).
* The report format should be well specified XML, with a documenting DTD,
for tool manipulation of the output data.
* Tools should exist to configurably format the XML reports into HTML
tables.
* Infrastructure has to support: Conformance testing, Performance testing
(including comparitive statistics), API Testing, Stress testing, Platform
testing, Smoke testing, Thread tests, and Exception handling testing.
* Testing has to be able to be run as a clean part of the build process.
* Test summary report for nightly mailing.

I'm sure there is lots more...  as I said, this is just off the top of my
head as a start.

-scott




---------------------------------------------------------------------
In case of troubles, e-mail:     webmaster@xml.apache.org
To unsubscribe, e-mail:          general-unsubscribe@xml.apache.org
For additional commands, e-mail: general-help@xml.apache.org

Mime
View raw message