db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel John Debrunner <...@apache.org>
Subject JUnit testing moving forward
Date Thu, 20 Jul 2006 19:22:22 GMT
We seem to have made great progess on the JUnit testing front, we have
some base classes with utility methods, about 40 Junit tests and a wiki
page describing Derby's JUnit use:

http://wiki.apache.org/db-derby/DerbyJUnitTesting

I think now would be a good time to establish some of the policies and
goals around JUnit in Derby. My current fear is that we are thinking too
much about getting JUnit tests to run in the existing harness, rather
than think of JUnit as the only harness moving forward.

As an example, we have eight JUnit tests in jdbcapi package, seven of
these are in the jdbcapi suite for the existing harness as *individual*
tests. If one run these eight Junit tests as a single JUnit suite then
the time taken is ~250 seconds, running them (all eight) as a derby
harness test suite takes ~500 seconds. Imagine if that performance
improvement was across all tests, derbyall would drop to two-three hours
on my laptop and around an hour on my desktop. Great savings for
everyone. So why not run these eight tests as a single JUnit suite from
the derby test harness suite, why force them to be individual tests?
Running the single JUnit suite in the derby harness took ~310 seconds.

My goal would be every test in Derby is a Junit test and the Derby
harness is removed. I think this will take some time and will be an
incremental process, but that should be the underlying thought when
working with tests.

I've been thinking that this goal would factor down into some
requirements/expectations/policies:

  - Any Derby Junit test will run sucessfully as a standalone test using
JUnit test runners (i.e. any public non-abstract class in Derby that
extends TestCase)

  - Every Derby package that contains JUnit tests has a class _Suite
that runs all the Junit tests in that package (see below for an
example). This can be directly for each JUnit test or through sub-suites.

  - Ideally this _Suite uses a single database for all the tests

  - Higher level suites would combine all the _Suites from the packages
resulting in something like derbyall

  - Some policy & utilities around cleanup and database removal.
     - utility methods to automatically clean the database (remove
tables etc.) based upon database meta data
     - utility method to remove the database and derby.log and anything else
     - assumption tests will leave a clean database and remove
everything once complete, but optionally leave the database for the next
 test (set by the calling suite?)

     - some clear policy on the configuration stuff, I couldn't see from
a quick look any guidance on how configurations are meant to work, e.g.
how would I set up tests to run under embedded and client, would this be
a high level suite that runs the jdbcapi._Suite test once in embedded
and then once as client?

There's probably a list of functions the current harness performs that
needs to be implemented in JUnit (as test decorators?) but these could
be tackled incrementally. My itch would be the security manager setup.
Would be good to get a list of these.

Thoughts, comments?
Dan.

-----
package org.apache.derbyTesting.functionTests.tests.jdbcapi;

import org.apache.derbyTesting.functionTests.util.BaseTestCase;

import junit.framework.Test;
import junit.framework.TestSuite;

public class _Suite extends BaseTestCase {
	
	public _Suite(String name)
	{
		super(name);
	}
	
	public static Test suite() {
	    TestSuite suite= new TestSuite();
	    suite.addTest(ProcedureTest.suite());
	    suite.addTestSuite(ScrollResultSetTest.class);
	    suite.addTest(ConcurrencyTest.suite());
	    suite.addTestSuite(HoldabilityTest.class);
	    suite.addTest(SURQueryMixTest.suite());
	    suite.addTest(SURTest.suite());
	    suite.addTestSuite(UpdateXXXTest.class);
	    suite.addTestSuite(URCoveringIndexTest.class);

	    return suite;
	}
}


Mime
View raw message