db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel John Debrunner <...@apache.org>
Subject [junit] different configurations at test level Was - Re: [junit] Top-level suites added
Date Thu, 28 Sep 2006 19:05:03 GMT
Vemund Ostgaard wrote:

> This would mean that when executing a single TestCase class (or suite) 
> with a junit runner, the default behaviour could be to run with all 
> (supported) frameworks if that is what is most common. Running with just 
> a selected framework could be possible with a system property.

I now think this is the correct direction for JUnit tests, so that 
running, say, the lang.TimeHandlingTest, would run it in embedded and 
network client, not just embedded. Thus there would not be a reliance on 
top-level suites to run in different fundamental configurations. I see 
fundamental configurations being embedded, derby client and possible the 
db2 client. Other configurations would be handed at a top-level suite, 
e.g. run a set of tests with an encrypted database.

However, I'm don't see a clear way forward of how to implement this.

Currently in suite() methods the code checks the current configuration 
and adds test cases based upon that information. e.g. only add this 
fixture if it's embedded. Thus the assumption is that the suite() method 
will be called multiple times, once per configuration.

To push the multiple configurations into the suite() I see a number of 
possible options;

A) Have a standard pattern of calling the suite method calling itself 
indirectly through existing decorator, 
TestConfiguration.derbyClientServerDecorator.

      public static Test suite() {
         ...
         // run under other configurations
         if (usingEmbedded()) {
            suite.add(
             TestConfiguration.derbyClientServerDecorator(
                   TimeHandlingTest.class);

            // possible similar code for DB2 client
         }
         ...

Seems a little confusing, because the embedded (base) configuration is 
the one that adds the other configurations, the usingEmbedded() is there 
to stop recursion.

Other issue is that if the test doesn't want to run in client, does it 
not include the standard pattern, or just use the usingXXX() methods to 
set up empty suites for the configuration it doesn't run?

B) Have a test's suite method just set up a suite as it sees fit:

       public static Test suite() {
           TestSuite suite = new TestSuite();

           // Embedded tests
           suite.add(TimeHandlingTest.class);
           suite.add("embeddedTestA");
           suite.add("embeddedTestB");
           suite.add("embeddedTestC");

           // Client tests
           TestSuite client = new TestSuite();
           client.add(TimeHandlingTest.class);
           client.add("clientTestA");

           suite.add(new NetworkServerTestSetup(client));

           return suite;
       }

Downsides are:
     possible lack of consistency, but maybe the consistency is use of 
the NetworkServerTestSetup decorator.
     how to handle remote server testing?

Upside is:
      clarity
      matches standard JUnit pattern
      non-reliance of having the configuration setup at suite setup time.

C) Something in the BaseTestClass that automatically runs the test 
fixtures in multiple configurations. e.g. modifying runBase to call 
super.runBase() for each configuration.

Upside is tests automatically run in all configurations.

Downside is that decision if to run a text fixture or not is based at 
test runtime and not suite setup time, which means cluttering test 
fixture methods with if statements to indicate if the will run or not.
Other downside is that a large number of tests may not need to run in 
multiple configurations, e.g. tests that test SQL functionality may not 
have much benefit running in any client mode.

-----------------------------------------------------

Are there any other ideas?  Having written them out, my preference would 
be B), seems very clear and straight forward. Before I wrote it out I 
was thinking that none of the solutions were good.

Thanks,
Dan.




Mime
View raw message