xml-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Davis" <...@us.ibm.com>
Subject Re: Test Infrastructure Project Proposal
Date Mon, 12 Feb 2001 22:32:07 GMT
I think you need to distinguish between what the tool can do
and what specific tests do.  For example you talk about the
types of tests (Conformance, API tests) but that's dependent
on how the tests are written.  I'd rather the list focus on the tool
itself and making sure that the test tool can run the tests that
do the things you mentioned.  Also, if you put things in the list
like:
  It must be able to have complex traversals of input directories.
could you explain these more?  It seems like you're talking about
a certain type of test for a certain project - so that line
is meaningless to others who don't know the project you might
be referring to.  Try to keep the list more generic, the
pluggable 'diff' program requirement is a good example.
Keeping it non-project specific will help ensure it's global use.
thanks,
-Dug


Scott_Boag@lotus.com on 02/12/2001 03:43:40 PM

Please respond to general@xml.apache.org

To:   general@xml.apache.org
cc:
Subject:  Re: Test Infrastructure Project Proposal




Here's a start on Test Infrastructure requirements, rather unstructured,
and off the top of my head:

* It must be able to have complex traversals of input directories.
* It should support easy running of categories of tests.
* A mechanism must exist to hand in datafiles to a single test program.
(In concrete, we need to run Xalan over some 1500 tests pairs of XSLT and
XML).
* Must have pluggable result comparitors (for instance, to compare XML
disregarding attribute order, to compare HTML disregarding non-significant
whitespace, etc.)
* Tests should be easily run stand-alone from the main infrastructure, for
easy debugging and problem isolation.  (This means, I don't like having to
subclass my test classes)
* Per-platform exclusions should be able to be specified (for instance,
known platforms that do not support certain encodings).
* The report format should be well specified XML, with a documenting DTD,
for tool manipulation of the output data.
* Tools should exist to configurably format the XML reports into HTML
tables.
* Infrastructure has to support: Conformance testing, Performance testing
(including comparitive statistics), API Testing, Stress testing, Platform
testing, Smoke testing, Thread tests, and Exception handling testing.
* Testing has to be able to be run as a clean part of the build process.
* Test summary report for nightly mailing.

I'm sure there is lots more...  as I said, this is just off the top of my
head as a start.

-scott




---------------------------------------------------------------------
In case of troubles, e-mail:     webmaster@xml.apache.org
To unsubscribe, e-mail:          general-unsubscribe@xml.apache.org
For additional commands, e-mail: general-help@xml.apache.org




Mime
View raw message