db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rick Hillegas (JIRA)" <j...@apache.org>
Subject [jira] Commented: (DERBY-2316) Convert compatibility/testScript.xml to JUnit
Date Wed, 28 Mar 2007 23:42:25 GMT

    [ https://issues.apache.org/jira/browse/DERBY-2316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12485029

Rick Hillegas commented on DERBY-2316:

Thanks for this improvement, Ole! Replacing the compatibiltiy driver with a JUnit program
will make it easier to add new vms and derby revs to the tested combinations. I have committed
this first slug of work at subversion revision 523505.

I have some suggestions for further improvements:

1) I found that I had to create a subdirectory named "system" in order for the tests to run.
I think that the test should do this for me. If for some reason that is not possible, then
I think that the javadoc should explain that this preliminary step is needed.

2) I think that the javadoc should explain that you can get verbose output by setting -Ddrb.tests.debug=true.
Later on, we should change this verbose flag to be the same one used by the other JUnit tests,
viz., derby.tests.debug.

3) It would be great to see some more defensive coding which fails the tests early if the
environmental variables in compatibilitytest.properties point at garbage. I believe that the
old ant harness does this kind of checking. I started out with a typo in the path to my junit.jar
and this caused the tests to fail and leave a directory of empty diagnostic files. It took
a while to figure out what was broken.

4) It would be nice if this test could follow the excellent pattern now used by the JUnitized
upgrade tests, i.e., require that the old releases live under a directory pointed to by derbyTesting.oldReleasePath.
That would eliminate a number of fragile variables in compatibilitytest.properties.


> Convert compatibility/testScript.xml to JUnit
> ---------------------------------------------
>                 Key: DERBY-2316
>                 URL: https://issues.apache.org/jira/browse/DERBY-2316
>             Project: Derby
>          Issue Type: Improvement
>          Components: Test
>    Affects Versions:
>         Environment: All
>            Reporter: Ole Solberg
>         Assigned To: Ole Solberg
>            Priority: Minor
>         Attachments: derby-2316_v1.diff.txt, derby-2316_v1.example_compatibilitytest.properties
> I have started converting compatibility/testScript.xml to JUnit to
> 1) be able to more dynamically specify which combinations to test, 
> 2) get standard JUnit reports from the test, and
> 3) more easily include the compatibility test in the regression test runs.
> I plan to use a property file (patterened after the current ant.property file for 
> the compatibility test), to specify jvm and derby library locations.
> With the growing number of jvm and derby versions I also think that it should be 
> possible to specify a number of different kinds of compatibility test combinations,
> for example:
> a) the current way, which is all combinations of derby and jvm on both 
>    server and client.                                            [(derbys*jvms)*(derbys*jvms)]
> b) Current trunk client and jvms  vs.  all server derbys and jvms. [(1*jvms)*(derbys*jvms)]
> c) All clients and jvms  vs.  current trunk server and jvms.        [(derbys*jvms)*(1*jvms)]
> d) Exact specification of the combinations to be tested.         [(N*M)*(X*Y)]
> Which kind of test to run should be specified in the property file.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message