db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ole Solberg <Ole.Solb...@Sun.COM>
Subject JUnit test infrastructure and Derby
Date Wed, 21 Sep 2005 18:30:32 GMT
We at the Sun Database Technology Group are exploring some ideas on
establishing a JUnit-based test infrastructure for Derby testing.
We would like to get your feedback on the feasibility of our approach.


There have already been discussions on derby-dev arguing that a test
infrastructure based on JUnit assertion testing is preferrable to the
existing canon-based testing.
For example
http://article.gmane.org/gmane.comp.apache.db.derby.devel/2143/match=junit

Lately, adding new JUnit based tests has been discussed:
http://mail-archives.apache.org/mod_mbox/db-derby-dev/200509.mbox/%3c4328336F.5060407@sun.com%3e


We expect a new JUnit based test infrastructure and the existing test
harness to co-exist for quite some time. Even if the goal would be to
convert all existing tests into equivalent JUnit tests that is not
something which can be done overnight.


Our initial focus for a new test infrastructure has been to be able to
perform Derby testing as efficiently as possible by exploiting a set of
computers which can be selected to act both as hosts for embedded Derby
as well as hosts for Derby servers and Derby (test) clients:
- The current test harness can only utilize a single machine, while our
solution includes being able to utilize multiple machines.
- The current test harness runs client/server tests on a single machine,
while our solution will make it possible to do true client/server tests
with a real network between the server and client(s).

The possibility of exploiting several machines for parallel testing will
also reduce the time it takes to run an equivalent of derbyall by
distributing the tests now run sequentially by derbyall. (See later for
a comment on wrapping the existing tests in JUnit.)


We will be able to distribute tests using embedded mode or local
client/server mode to a number of single hosts to exploit parallel
testing and thus speed up the test cycle.
Distributed testing using server and clients on several hosts is supported.
The default "test lab" will be a single host ("localhost") which is
equivalent to today's "test lab" for running the 'derbyall' regression
test etc. I.e. the test infrastructure will support todays regression
testing on single hosts "out of the box".
Tests will be able to check whether the test lab available can support
the test functionality, for example whether client and server can be
distributed to different hosts to support client/server protocol testing.

To be able to report test results in different formats (for example pure
text on a file, or to a test report database) we will define a reporting
interface.


Another important goal for us is that the test infrastructure shall be
supportive of performance testing, robustness testing, adding background
load on the database, and other non-functional testing. This is, at
least partly, supported by the mechanisms for distributing clients and
server on different hosts.



We believe that an implementation based on an extension of the JUnit
'TestCase' will provide the flexibility needed to support the
distribution of server and clients. The extended class,
'DerbyJUnitTestCase', reads "test lab" information from a property file.

We have also done some work on wrapping the existing Derby tests into a
JUnit framework. We have started from Dan Debrunner's 'ijRunner.java'
(http://article.gmane.org/gmane.comp.apache.db.derby.devel/2170) and
have managed to have some .sql and .java tests in the derbylang
(.../functionTests/tests/lang/) suite run successfully using our wrapper
classes.



We also plan to see if there are mechanisms in the existing test
harness which can be reused in a JUnit based test infrastructure.
For example. the existing test harness knows of supported jvms (jdk*,
ibm*, j9_*) and this JUnit based test harness would need equivalent info
to select/un-select jvm dependent tests. The same would be the case for
the Derby framework(or "mode"), i.e. Embedded, DerbyNetClient and
DerbyNet(IBM Universal JDBC Driver).






-- 
Regards
Ole Solberg, Database Technology Group,
Sun Microsystems, Trondheim, Norway



Mime
View raw message