db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vemund Ostgaard <Vemund.Ostga...@Sun.COM>
Subject Re: Running the junit testsuite with a remote server
Date Tue, 18 Sep 2007 09:11:39 GMT
Oops, hit the send button by accident in the middle of writing, so 
resending this mail.

Myrna van Lunteren wrote:
> Starting to reason from the tests, I think part of the issue is that
> there are tests that test the correct starting and shutting down of
> the network server; or that need the server to be started with
> specific properties as opposed to the 'normal' mode. And as far as I
> know we cannot start and shutdown a server on a remote machine, and
> thus these tests cannot run with a remote server.
So there is a group of tests that can only be run with the server on the
localhost and/or only make sense to run with the server on the localhost.

Probably there is also a group of tests that only make sense to run with
the server on a remote machine, for instance verifying that you are not
allowed to shut it down?

Finally there is the group of tests that make sense to run both with a
remote server and with a local server, and possibly also in embedded mode.
> Theoretically, all tests that do not need to test starting/shutting
> down, or that do not test the server with specific properties set,
> should, be 'runnable' with remote server that is already running.
> However, is that really necessary? What things could go wrong when
> connecting to a remote server versus a server on localhost? I would
> think mostly things to do with connections, permissions,
> import/export. So, maybe it makes sense after all to have a dedicated
> set of tests for remote server testing.
 From reading about the junit tests on the wiki and taking a look at the
code, I understand that the logic for controlling which test fixtures
are run with an embedded database and which fixtures are run with a
network server, is in the suite() method of the various test classes. I
guess this is the place to put logic that will give you a suite with the 
right fixtures in it and with the right decorators, depending on whether 
you want to run tests with a local server, a remote server, or embedded.

It should be possible to run only the fixtures for the modes that you 
want to test, but ideally it should also be possible to run all the 
fixtures in all the modes that they support. So I guess what is needed 
in terms of functionality for the test framework is to be able to say
* I do/don't want to run the "embedded tests"
* I do/don't want to run the "local server tests"
* I do/don't want to run the "remote server tests"

The default behavior could be as it is today, to run the tests for all 
modes, provided that the framework has been instructed about the 
whereabouts of a remote server. If not, skip the "remote server tests".

> Note that with remote server testing, trouble brews in the area of
> cleanup; I believe the current junit framework attempts to delete
> files that get created by the server; and I think deleting of files on
> a remote machine should be impossible. I wasted considerable time with
> the old test harness trying to identify tests that would run cleanly
> in sequence; only to find that a previously working set was made to
> fail because a uncooperative test/testcase was added.

Is it necessary to delete files to be able to clean properly between 
test fixtures? It wouldn't be sufficient in most cases to just drop 
everything with SQL?

  It may be better
> to create a suite of tests, normally part of suites.All, which
> can/should be run against a remote server as well.

I am concerned about a top level suite knowing about which tests are 
supposed to run in this or that mode. I think the way it is now, that 
this knowledge is contained in each testcase class, makes sense.
> For the desire to be able to run all client tests; that is more an
> efficiency issue useful for someone working on client only changes;
> when one is certain there is no need to run the tests with embedded. I
> think maybe a class that travels through the test suites and
> identifies the ones that run with DerbyNetClient would cover that
> requirement. The converse could also be true (run only embedded
> tests), but I believe this can be achieved by excluding
> derbyclient.jar from the classpath.

Another reason to be able to run only the "embedded tests" or "client 
tests" would be to limit which jars/classes that are in the classpath to 
the ones that are strictly required in that mode and test that nothing 
breaks because of that.


View raw message