db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Van Couvering <David.Vancouver...@Sun.COM>
Subject Re: JUnit testing moving forward
Date Thu, 20 Jul 2006 21:29:34 GMT
I think this is a laudable goal.  I agree it would need to be done 
incrementally.  I think it would really help to have a Wiki page where 
work around this can be clarified and coordinated.  Probably a JIRA to 
hang subtasks off of too.

Here are some questions I have around this

- does moving to a JUnit harness imply removing canon-based testing and 
only using assertion-based testing?  Or is that a separate, orthogonal 
goal (one I which I think is important, both for master file management 
overhead as well as improving our ability to truly test backward 
compatibility without having to sift through unhelpful diffs).

- Where does the responsibility lie to manage different results in 
different environments?

I'm sure I'll have more questions, but those are the ones that come up 
for me right now...



Daniel John Debrunner wrote:
> We seem to have made great progess on the JUnit testing front, we have
> some base classes with utility methods, about 40 Junit tests and a wiki
> page describing Derby's JUnit use:
> http://wiki.apache.org/db-derby/DerbyJUnitTesting
> I think now would be a good time to establish some of the policies and
> goals around JUnit in Derby. My current fear is that we are thinking too
> much about getting JUnit tests to run in the existing harness, rather
> than think of JUnit as the only harness moving forward.
> As an example, we have eight JUnit tests in jdbcapi package, seven of
> these are in the jdbcapi suite for the existing harness as *individual*
> tests. If one run these eight Junit tests as a single JUnit suite then
> the time taken is ~250 seconds, running them (all eight) as a derby
> harness test suite takes ~500 seconds. Imagine if that performance
> improvement was across all tests, derbyall would drop to two-three hours
> on my laptop and around an hour on my desktop. Great savings for
> everyone. So why not run these eight tests as a single JUnit suite from
> the derby test harness suite, why force them to be individual tests?
> Running the single JUnit suite in the derby harness took ~310 seconds.
> My goal would be every test in Derby is a Junit test and the Derby
> harness is removed. I think this will take some time and will be an
> incremental process, but that should be the underlying thought when
> working with tests.
> I've been thinking that this goal would factor down into some
> requirements/expectations/policies:
>   - Any Derby Junit test will run sucessfully as a standalone test using
> JUnit test runners (i.e. any public non-abstract class in Derby that
> extends TestCase)
>   - Every Derby package that contains JUnit tests has a class _Suite
> that runs all the Junit tests in that package (see below for an
> example). This can be directly for each JUnit test or through sub-suites.
>   - Ideally this _Suite uses a single database for all the tests
>   - Higher level suites would combine all the _Suites from the packages
> resulting in something like derbyall
>   - Some policy & utilities around cleanup and database removal.
>      - utility methods to automatically clean the database (remove
> tables etc.) based upon database meta data
>      - utility method to remove the database and derby.log and anything else
>      - assumption tests will leave a clean database and remove
> everything once complete, but optionally leave the database for the next
>  test (set by the calling suite?)
>      - some clear policy on the configuration stuff, I couldn't see from
> a quick look any guidance on how configurations are meant to work, e.g.
> how would I set up tests to run under embedded and client, would this be
> a high level suite that runs the jdbcapi._Suite test once in embedded
> and then once as client?
> There's probably a list of functions the current harness performs that
> needs to be implemented in JUnit (as test decorators?) but these could
> be tackled incrementally. My itch would be the security manager setup.
> Would be good to get a list of these.
> Thoughts, comments?
> Dan.
> -----
> package org.apache.derbyTesting.functionTests.tests.jdbcapi;
> import org.apache.derbyTesting.functionTests.util.BaseTestCase;
> import junit.framework.Test;
> import junit.framework.TestSuite;
> public class _Suite extends BaseTestCase {
> 	public _Suite(String name)
> 	{
> 		super(name);
> 	}
> 	public static Test suite() {
> 	    TestSuite suite= new TestSuite();
> 	    suite.addTest(ProcedureTest.suite());
> 	    suite.addTestSuite(ScrollResultSetTest.class);
> 	    suite.addTest(ConcurrencyTest.suite());
> 	    suite.addTestSuite(HoldabilityTest.class);
> 	    suite.addTest(SURQueryMixTest.suite());
> 	    suite.addTest(SURTest.suite());
> 	    suite.addTestSuite(UpdateXXXTest.class);
> 	    suite.addTestSuite(URCoveringIndexTest.class);
> 	    return suite;
> 	}
> }

View raw message