db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rick Hillegas <Richard.Hille...@Sun.COM>
Subject Re: JUnit testing moving forward
Date Thu, 20 Jul 2006 20:16:48 GMT
There are lots of good suggestions in this message. I would add the 
following: If we're going to consider a new test harness, then we could 
evaluate using Ant to drive the junit tests. Ant already has an optional 
<junit> task for this purpose.


Daniel John Debrunner wrote:

>We seem to have made great progess on the JUnit testing front, we have
>some base classes with utility methods, about 40 Junit tests and a wiki
>page describing Derby's JUnit use:
>I think now would be a good time to establish some of the policies and
>goals around JUnit in Derby. My current fear is that we are thinking too
>much about getting JUnit tests to run in the existing harness, rather
>than think of JUnit as the only harness moving forward.
>As an example, we have eight JUnit tests in jdbcapi package, seven of
>these are in the jdbcapi suite for the existing harness as *individual*
>tests. If one run these eight Junit tests as a single JUnit suite then
>the time taken is ~250 seconds, running them (all eight) as a derby
>harness test suite takes ~500 seconds. Imagine if that performance
>improvement was across all tests, derbyall would drop to two-three hours
>on my laptop and around an hour on my desktop. Great savings for
>everyone. So why not run these eight tests as a single JUnit suite from
>the derby test harness suite, why force them to be individual tests?
>Running the single JUnit suite in the derby harness took ~310 seconds.
>My goal would be every test in Derby is a Junit test and the Derby
>harness is removed. I think this will take some time and will be an
>incremental process, but that should be the underlying thought when
>working with tests.
>I've been thinking that this goal would factor down into some
>  - Any Derby Junit test will run sucessfully as a standalone test using
>JUnit test runners (i.e. any public non-abstract class in Derby that
>extends TestCase)
>  - Every Derby package that contains JUnit tests has a class _Suite
>that runs all the Junit tests in that package (see below for an
>example). This can be directly for each JUnit test or through sub-suites.
>  - Ideally this _Suite uses a single database for all the tests
>  - Higher level suites would combine all the _Suites from the packages
>resulting in something like derbyall
>  - Some policy & utilities around cleanup and database removal.
>     - utility methods to automatically clean the database (remove
>tables etc.) based upon database meta data
>     - utility method to remove the database and derby.log and anything else
>     - assumption tests will leave a clean database and remove
>everything once complete, but optionally leave the database for the next
> test (set by the calling suite?)
>     - some clear policy on the configuration stuff, I couldn't see from
>a quick look any guidance on how configurations are meant to work, e.g.
>how would I set up tests to run under embedded and client, would this be
>a high level suite that runs the jdbcapi._Suite test once in embedded
>and then once as client?
>There's probably a list of functions the current harness performs that
>needs to be implemented in JUnit (as test decorators?) but these could
>be tackled incrementally. My itch would be the security manager setup.
>Would be good to get a list of these.
>Thoughts, comments?
>package org.apache.derbyTesting.functionTests.tests.jdbcapi;
>import org.apache.derbyTesting.functionTests.util.BaseTestCase;
>import junit.framework.Test;
>import junit.framework.TestSuite;
>public class _Suite extends BaseTestCase {
>	public _Suite(String name)
>	{
>		super(name);
>	}
>	public static Test suite() {
>	    TestSuite suite= new TestSuite();
>	    suite.addTest(ProcedureTest.suite());
>	    suite.addTestSuite(ScrollResultSetTest.class);
>	    suite.addTest(ConcurrencyTest.suite());
>	    suite.addTestSuite(HoldabilityTest.class);
>	    suite.addTest(SURQueryMixTest.suite());
>	    suite.addTest(SURTest.suite());
>	    suite.addTestSuite(UpdateXXXTest.class);
>	    suite.addTestSuite(URCoveringIndexTest.class);
>	    return suite;
>	}

View raw message