db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kristian Waagan <Kristian.Waa...@Sun.COM>
Subject Features of the JUnit test execution harness
Date Tue, 24 Jan 2006 14:07:52 GMT

I have wanted to rewrite a old canon based test to a JUnit test for a 
while, but each time I get started I always run into issues that we do 
not have guidelines for yet. So, I have a few questions I would like 
some feedback on.
I know there are a lot of issues to consider, far more then the one I 
mention here, but I hope to get a feeling of the general direction we 
want to move in.

1) Should we support specific ordering of tests (within a single class)?

This feature is wanted if a number of tests operate on shared data and 
each test method can be seen as a step in the test. If the order is 
mixed up, the intermediate results will not be as expected and all the 
tests may fail.
In general, assuming anything about the execution order of tests is 
strongly disencouraged. However, since we are testing a database system, 
we might want to allow for this. This feature can influence the both the 
size of the test methods and the execution time for the test/suite.

2) Should we support setting up a shared test fixture for a set of tests?

This is a common issue with JUnit, and there are mechanisms to handle 
it. For instance, we could let tests that require this to wrap itself in 
a TestSetup instance (as described in the JUNit FAQ). Letting 
DerbyJUnitTest extend TestSetup is another solution, but it might have 
unwanted side-effects.

3) Many tests just want a database to work on. Other tests want to 
create their own database due to specific requirements. How do we make 
this easy for the developer?

4) Should it be possible to run single JUnit test/suite from the command 
line by simply invoking the main method?

Personally, I think this should be possible. This require us to remove 
the current use of main for running a test in the old test harness, 
which I believe is doable with a few changes (in the old harness).

5) Is the ultimate goal to end up with a pure JUnit based test harness?

If this is the goal, we should design for this and accept some extra 
work for plugging JUnit tests into the old test harness.

6) Where, and how, do we want to store settings for a test?

7) How is it determined which framework(s) a JUnit test should be run 
in? (i.e. embedded, client, JCC)

If you have other opinions or comments related to JUnit testing, please 
put them forth :)

I hope we are able to get a discussion on all or some of these issues, 
and that it results in a Wiki page describing JUnit testing for Derby. I 
created a page here: http://wiki.apache.org/db-derby/DerbyJUnitTesting


View raw message