xml-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Scott_B...@lotus.com
Subject Re: Test Infrastructure Project Proposal
Date Mon, 12 Feb 2001 20:43:40 GMT

Here's a start on Test Infrastructure requirements, rather unstructured,
and off the top of my head:

* It must be able to have complex traversals of input directories.
* It should support easy running of categories of tests.
* A mechanism must exist to hand in datafiles to a single test program.
(In concrete, we need to run Xalan over some 1500 tests pairs of XSLT and
XML).
* Must have pluggable result comparitors (for instance, to compare XML
disregarding attribute order, to compare HTML disregarding non-significant
whitespace, etc.)
* Tests should be easily run stand-alone from the main infrastructure, for
easy debugging and problem isolation.  (This means, I don't like having to
subclass my test classes)
* Per-platform exclusions should be able to be specified (for instance,
known platforms that do not support certain encodings).
* The report format should be well specified XML, with a documenting DTD,
for tool manipulation of the output data.
* Tools should exist to configurably format the XML reports into HTML
tables.
* Infrastructure has to support: Conformance testing, Performance testing
(including comparitive statistics), API Testing, Stress testing, Platform
testing, Smoke testing, Thread tests, and Exception handling testing.
* Testing has to be able to be run as a clean part of the build process.
* Test summary report for nightly mailing.

I'm sure there is lots more...  as I said, this is just off the top of my
head as a start.

-scott




Mime
View raw message