db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "John H. Embretsen (JIRA)" <derby-...@db.apache.org>
Subject [jira] Commented: (DERBY-1141) test harness usage of useprocess needs improvement
Date Tue, 18 Apr 2006 13:50:18 GMT
    [ http://issues.apache.org/jira/browse/DERBY-1141?page=comments#action_12374941 ] 

John H. Embretsen commented on DERBY-1141:
------------------------------------------

I think it is a good thing getting more tests/suites to run with useprocess=false, as this
may improve running time and help debugging, among other things. So I definitely think this
work has value for the Derby community.

I have not studied in detail all aspects of 'DERBY-1141_20060413.diff' (henceforth "the patch"),
but I have a couple of minor comments - mostly comments about the comments, actually...


Starting/stopping network server:
-----------------------------------------------------------

I don't think it is clear from the comments in the patch in which cases the network server
is started or stopped, or not - and the reason for doing so. 

For example, in RunList.java the patch adds:

+        // If useprocess is false, and this is a networkserver test,
+        // make sure we shutdown network server

I think it would be good to include a comment on _why_ we have to make sure to shut down the
network server.

In RunTest.java (main()) the patch adds:

+            // if this is a suite, and we're using useprocess, actually don't need
+            // to start networkserver if it's running
+            if ((!useprocess) && (isSuiteRun))
+            {

It is probably a good idea to be consistent when referring to the useprocess flag. For example,
I don't think it is clear what "we're using useprocess" actually means.
I see from the code that what is meant is useprocess=false, while I would interpret "using
useprocess" as useprocess=true. Please include "true" or "false" when referring
to a specific value of the useprocess flag.

Here (in the same part of the patch) I also think it would be nice to include short comments
about why (or why not) the server is started or stopped, although this may be derived from
the code and older comments elsewhere in RunTest.java. The test harness is hard enough to
understand as it is, so more and better comments are always appreciated (by me, at least).


JUnit and useprocess=false:
----------------------------------------------------

Some may be surprised to find that junit tests are run in a separate process by default (instead
of being skipped) when useprocess=false. However, I think I see some advantages of doing it
this way, so I accept (a bit reluctantly, perhaps) this behavior for now.

In the patch I also find:

-        if ( useprocess )
+	    // Always create a process if testType is junit, otherwise testRunner
+	    // would exit a suite prematurely, and we can't debug that anyway and
+	    // they should run quick anyway.
+        if ( useprocess || (!useprocess && testType.equals("junit")))

I'm not sure I understand/remember exactly why junit tests cannot be run with useprocess=false
(i.e. why "testRunner" (I assume this means junit.*ui.TestRunner) would exit a suite prematurely).
Feel free to educate me on this subject (otherwise I guess I'll look into it myself when time
permits) ;)


> test harness usage of useprocess needs improvement
> --------------------------------------------------
>
>          Key: DERBY-1141
>          URL: http://issues.apache.org/jira/browse/DERBY-1141
>      Project: Derby
>         Type: Improvement

>   Components: Test
>     Reporter: Myrna van Lunteren
>     Assignee: Myrna van Lunteren
>  Attachments: DERBY-1141_20060413.diff, DERBY-1141_20060413.stat
>
> The 'old' functionTests harness has property useprocess, which, when set to false, causes
tests within a suite to run from within one jvm, thus creating databases and files in the
same directory.
> This functionality is useful for debugging test runs within an IDE.
> Also, it is potentially useful for speeding up testruns.
> However, currently, there are some problems and shortcomings of this functionality.
> - For instance, when running a networkserver test with framework DerbyNet of type .java
one sees the message 'Security Manager not installed' show up within the test output, thus
causing the test to fail.
> - running derbynet(client)mats suite with useprocess=false doesn't actually make things
any faster,  because networkserver gets started fresh for every test. 
>   However, it should be feasible to start networkserver with the first test in a suite,
then use that 
>   networkserver  for subsequent tests and shutdown networkserver when the suite finishes.
> - Also, (reported by Mike Matrigali off-list) currently, the following:
> ------------------
> java -Duseprocess=false -Doutputdir=<snippeddirname>/newout  -Dkeepfiles=true org.apache.derbyTesting.functionTests.harness.RunTest
lang/closed.java
> ------------------
> makes the test fall out of the harness with: Exception in thread "main"
> and in the .tmp:
> ------------------
> java.security.AccessControlException: access denied (java.io.FilePermission <snippeddirname>\newout
read)
>         at java.security.AccessControlContext.checkPermission(AccessControlContext.java:269)
>         at java.security.AccessController.checkPermission(AccessController.java:401)
>         at java.lang.SecurityManager.checkPermission(SecurityManager.java:524)
>         at java.lang.SecurityManager.checkRead(SecurityManager.java:863)
>         at java.io.File.exists(File.java:678)
>         at java.io.Win32FileSystem.canonicalize(Win32FileSystem.java:360)
>         at java.io.File.getCanonicalPath(File.java:513)
>         at org.apache.derbyTesting.functionTests.harness.RunTest.execTestNoProcess(RunTest.java:2370)
>         at org.apache.derbyTesting.functionTests.harness.RunTest.testRun(RunTest.java:443)
>         at org.apache.derbyTesting.functionTests.harness.RunTest.main(RunTest.java:302)
> --------------------------------
>   removing the -Doutputdir gets around that problem, but shouldn't be needed.
> - there is the reference http://www.nabble.com/AccessControlException-when-running-functional-tests-t1321374.html#a3526947
>   I couldn't duplicate this yet, maybe more info to follow.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira


Mime
View raw message