db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Myrna van Lunteren <m.v.lunte...@gmail.com>
Subject Re: Test Harness useprocess=false question
Date Fri, 18 Nov 2005 19:31:28 GMT
On 11/18/05, John Embretsen <John.Embretsen@sun.com> wrote:
> Myrna van Lunteren wrote:
> > On 11/17/05, *Daniel John Debrunner* <djd@debrunners.com
> > <mailto:djd@debrunners.com>> wrote:
> >
> > The test harness can run tests within the same jvm if useprocess=false.
> > This is set for the nist suite, so that all the sub-scripts are run
> > in a
> > single JVM, mainly for performance reasons.
> >
> [snip]
> >
> > This also implies that any tests that need special flags cannot be run
> > with useprocess = false.
> Are there other cases (other than the nist tests) where we should be
> able to run with useprocess set to false?

  To be frank, a former Cloudscape QA had at one point made it his goal to
clean up and make as many tests as possible run with useprocess=false.
However, that person was sent packing somewhere in the take-overs and
lay-off rounds (first by Informix, then by IBM) before finishing that
project. And it has not been high on anyones list since.
 I think most tests *ought* to be able to run with useprocess set to false.
However, this requires a certain mindset of the test writer - as it involves
leaving the database behind exactly as you found it; or if absolutely
required, use identifiers that are not used anywhere else in the tests. This
is not always the case - we happily rely on the test harness cleaning up
after us, and it's hard to think of imaginative name for a test table with
an integer and character column.
 As I need to use useprocess for DERBY-413, I am cleaning up the tests in
derbynetclientmats (which includes jdbcapi, jdbc20 and jdk14 suites), but
I'm not going beyond.
 So, in short, as you found the only tests that are currently running with
useprocess=false are nist.

I dug into some of the test harness code a while ago, trying to
> understand a little better how the machinery works (I won't say anything
> about how successful I was... ;) ).

 :-) well, it works after a fashion...sort of. The whole thing could do with
a clean-up, but it seems more sensible to spend time on writing JUnit tests,
at this point.

 I think I found other issues with
> useprocess=false...

 Very likely.

For example, there are a number of tests that include a call to
> System.exit(), which means that when useprocess=false, the test shuts
> down the JVM, and the harness is not able to process the results and
> clean up after itself.
> README.htm in the java/testing directory states the following:
> <Quote>
> useprocess
> (default=true) Controls whether RunTest runs the test in a separate VM
> or in a thread in harness VM. It is potentially useful for debugging
> tests. Unit tests are not (yet) runnable with "useprocess=false", though.
> <end Quote>
> It does not say anything about which tests "useprocess=false" may or may
> not work with, other than unit tests.
> So I guess the question is: Is this a bug? If so, is it a documentation
> bug or a bug in each of the tests listed in the attachment to this e-mail?

 Not a bug perse, but if someone wants, they can open a jira enhancement &
start making the rest of the tests (not including derbynetmats,
derbynetclientmats, jdbcapi, jdbc20, jdk14 and multi, 'cause I'm doing
those) run ok.

> John
> Executing: find
> ../clean/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/
> -print | xargs grep -n 'System.exit(' | grep -v '.svn'
> ../clean/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/lang/SpillHash.java:104:
> System.exit(0);
> ../clean/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/lang/SpillHash.java:109:
> System.exit(1);
> ../clean/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/lang/timestampArith.java:312:
> System.exit(1);
> ../clean/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/lang/timestampArith.java:358:
> System.exit(1);
> ../clean/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/lang/timestampArith.java:364:
> System.exit(1);
> ../clean/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/lang/timestampArith.java:366:
> System.exit(0);
> ../clean/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/lang/CharUTF8.java:118:
> System.exit(-1);
> ../clean/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/lang/longStringColumn.java:118:
> System.exit(-1);
> ../clean/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/derbynet/testSecMec.java:81:
> System.exit(-1);
> ../clean/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/derbynet/DerbyNetAutoStart.java:98:
> System.exit(0);
> ../clean/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/derbynet/DerbyNetAutoStart.java:103:
> System.exit(1);
> ../clean/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/derbynet/DerbyNetAutoStart.java:149:
> System.exit(0);
> ../clean/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/derbynet/DerbyNetAutoStart.java:171:
> System.exit(1);
> ../clean/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/derbynet/DerbyNetAutoStart.java:246:
> System.exit(1);
> ../clean/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/derbynet/dataSourcePermissions_net.java:87:
> System.exit(-1);
> ../clean/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/store/TestDiskHashtable.java:93:
> System.exit( (failed == 0) ? 0 : 1);

View raw message