db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vemund Ostgaard <Vemund.Ostga...@Sun.COM>
Subject Re: [junit] Top-level suites added
Date Wed, 30 Aug 2006 12:52:50 GMT


Daniel John Debrunner wrote:

>I added some top-level suites that are intended to be run directly by
>JUnit runners, not via the test harness. They are in the package:
>
>org.apache.derbyTesting.functionTests.suites
>
>AllPackages - A suite of all the _Suite suites for the function test
>packages
>
>Embedded - All the tests for embedded. Currently just includes
>AllPackages.suite(), but future may add more in different
>configurations, e.g. encryption.
>
>Client - All the tests for client, runs the AllPackages.suite() with a
>decorator that changes the configuration and boots the network server
>(using the existing network server decorator)
>
>All - All the tests, currently includes Client.suite() and
>Embedded.suite(). (Eventual replacement for derbyall)
>
>It's not all working fine yet, a couple of issues:
>
>  1) The network server decorator in jdbcapi._Suite clashes with the one
>in Client
>  
>
I think that adding the network server decorater to a test should just 
be a way of saying that this test needs a server running, not knowing 
wether it might allready be running. If it is not running, start it. If 
it allready runs, leave it be.

>  2) Some tests fail in the Client suite, need to investigate.
>Additional tests were failing due to some test using the old checks to
>see if it was running embedded or not, rather than the utilities
>provided in the JUnit classes.
>
>I've checked these in for others to see the direction I'm proposing,
>with the goal being all tests running under JUnit with no harness.
>
>Example of running these:
>
>java -Dderby.system.home=${PWD} junit.textui.TestRunner
>          org.apache.derbyTesting.functionTests.suites.Embedded
>
>java -Dderby.system.home=${PWD} junit.textui.TestRunner
>          org.apache.derbyTesting.functionTests.suites.Client
>
>java -Dderby.system.home=${PWD} junit.textui.TestRunner
>          org.apache.derbyTesting.functionTests.suites.All
>
>Comments?
>  
>
I have a suggestion, based on the following view:

* I would avoid creating suites just to handle the various frameworks, 
or other configuration related things. This would mean a simpler 
structure of suites, easier to understand and maintain. If we want to 
have a  MATS suite for instance, we would not need an embedded_MATS 
suite and a client_MATS suite and so forth to build the structure.
* It would be good if as little as possible (ideally no) magic was 
performed on a suite level. Tests should behave the same way when 
executed through the lowest level "leaf" suite as when executed as part 
of "derbyall", making it easier to reproduce problems.

How could this be accomplished?

Handle everything related to frameworks and configuration on a "leaf" 
level, I believe this could be done in the suite() method in each 
TestCase. Just add each Test once for each framework that this run is 
configured to use, using the correct decorator. If a test is not 
supposed to be run for a given framework, that information should be 
maintained in that TestCase in some way, so that test is not added for 
that framework. Default would be to run all Tests in all frameworks.

This would mean that when executing a single TestCase class (or suite) 
with a junit runner, the default behaviour could be to run with all 
(supported) frameworks if that is what is most common. Running with just 
a selected framework could be possible with a system property.

Vemund


Mime
View raw message