db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vemund Ostgaard <Vemund.Ostga...@Sun.COM>
Subject Re: [Running] Top-level junit tests - WAS Re: [jira] Commented: (DERBY-3316) Leak in client if ResultSet not closed
Date Thu, 24 Jan 2008 13:01:45 GMT
Daniel John Debrunner wrote:
> Myrna van Lunteren wrote:
>> I want to point out that to run platform tests it's not always
>> possible to use ant - not all OSs or machines I try to run on during a
>> platform test have ant available.
>> So if another solution can be found than relying on junit-all I'd 
>> appreciate it.
> It's not really relying on ant, the junit-all target just provides a 
> mechanism to run a set of tests, just as suites.All provides a 
> mechanism to run almost the same set of tests.
I think a problem here is that the naming implies that both "junit-all" 
and "suites.All" will run all the tests, at least that has been my own 
misconception. Apparently "ant junit-all" runs some tests that are not 
run by "junit suites.All", and I also think "junit suites.All" runs some 
tests that are not run by "ant junit-all". Because of this there is no 
easy way to run all the tests, and it is not trivial to figure out which 
tests gets run when I use this target but not when I use that. Ideally 
it should be easy for someone that just joined the community to figure 
out how to run all the tests, to qualify Derby on a different OS or jvm 
for instance.
> There's no requirement for anyone to use ant junit-all or suites.All.
> In addition there's nothing stopping anyone from contributing another 
> mechanism to run tests, e.g. a script file, a different grouping of 
> tests etc.
> The advantage of using ant or a Junit suite is that they are standard 
> mechanisms, and thus the tests can be run by other tools easily, such 
> as Elipse, junit test runners or cruise control.
I agree, this is a good thing.

Still I think it would be valuable to make clear if two mechanisms for 
running the tests are intended to be equivalent, and if so try to 
synchronize their behavior as much as possible. For instance, I believe 
the "ant junit-all" mechanism requires significantly less memory for the 
jvm than "junit suites.All" does, as it executes the different subsuites 
in suites.All in different jvms. I could add a new test and run it 
successfully with "ant junit-all", but it might break for anyone using 
"junit suites.All" because of different memory constraints. If I 
contribute a patch I would not want the mechanism I use to run the tests 
possibly hide problems that will hit others that run the same tests but 
with a different mechanism, to the extent possible.

So, I think it would be good to work towards tests being executed in 
environments that are as similar as possible, and in the same order, 
etc. when using different mechanisms that are intended to be equivalent.


View raw message