On 1/31/06, Kristian Waagan <Kristian.Waagan@sun.com> wrote:
Differences in output should be irrelevant. Although not what you
mentioned above, the issue of (execution) control is very relevant. The
logic for running the tests multiple times, each time with a different
setup/environment must be located somewhere. I think Andreas' proposal
of introducing a separate JUnit test type (see
http://www.nabble.com/running-JUnit-tests-t887682.html#a2300670) makes
sense, as it gives us more freedom w.r.t. handling of JUnit tests.
Yes, that proposal made sense to me. I personally like the approach of having a class for various/different configurations. Although that could get out of hand.
Does this 'throw away' the work that Rick is doing on DERBY-874?
Following Andreas' approach we'd still be able to run the individual tests separately, yes?
Thanks for the answer Myrna. And good work on the readme!  
credit due: many people have reviewed, and improved it...
Right now I feel that it is very hard to start writing a new JUnit
harness, since the old harness seem to contain a lot of functionality
and "magic". I wonder if it would be best to just start
converting/writing some tests and create a framework along the way.
Yes, a new harness is hard, and yes, the old harness has been made to adapt to many different needs...and we should take those into account. (I'll add some thoughts on that wiki soon). But the old harness has grown unwieldy and hard to maintain and is slow. To take what we learned and craft it onto something new will be better, though daunting.  
Some ideas of Dan's are in this thread from about a year ago:

The above approach would probably result in several major rewritings of
"the new framework" and the existing JUnit tests, but it might be better
than thinking out everything at once (that's hard!). Perhaps I will
rewrite an old test I have been working a little on, and put it out for
review and comments, unless someone else beats me to it ;)
Personally, I do better when I have code to prod and modify through iterations...
I'd be interested to review.