Mailing-List: contact derby-dev-help@db.apache.org; run by ezmlm
Precedence: bulk
Reply-To: <derby-dev@db.apache.org>
Message-ID: <70826722.1144958643070.JavaMail.jira@brutus>
Date: Thu, 13 Apr 2006 20:04:03 +0000 (GMT+00:00)
From: "Myrna van Lunteren (JIRA)" <derby-dev@db.apache.org>
To: derby-dev@db.apache.org
Subject: [jira] Updated: (DERBY-1141) test harness usage of useprocess needs
 improvement
In-Reply-To: <1484019075.1143051788254.JavaMail.jira@ajax>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

     [ http://issues.apache.org/jira/browse/DERBY-1141?page=all ]

Myrna van Lunteren updated DERBY-1141:
--------------------------------------

    Attachment: DERBY-1141_20060413.stat
                DERBY-1141_20060413.diff

Here is a patch that allows one to run other suites than nist, especially tests using networkserver, to run with useprocess=false.

There are various issues, and I have not tested all suites...
- I have only tested derbynetclientmats, derbynetmats,  jdbcapi, jdbc20, jdk14, multi. 
- my last test run was a while ago, any new tests might cause failures...
- tests that failed when run those suites with useprocess=false are listed in <framework>Useprocess.exclude files in the functionTests/suites dir. Tests will get skipped when they're run in a suite && useprocess==false && they're listed in the matching <framework>Useprocess.exclude file.
- the useprocess results in a different location for some files, as a result, a number of tests that depend on the 'normal' process will fail. 
- tests use the same database, so if they don't run clean, subsequent tests will fail. 
- none of the junit tests work with useprocess false.

On the positive side:
- the suites do run faster...(even ignoring the fact that quite a few tests get excluded especially for Derby client).
- the using of the same database is similar to how the RemoteServer testing works, and thus, running things this way makes the maintenance for that setup a less lonely experience.  (  :-)  )
- this patch did coincidentally fix Mike's report of failure with -Doutput.
- this patch in passing stops the harness from setting user.dir. (https://issues.apache.org/jira/browse/DERBY-577)

I had at one point separate suites of tests that worked with useprocess and not, but I found it got to be too confusing, with too many suites...

One change to note in this is that networkserver does not get stopped and started for every test if useprocess=false. 

This is a work in progress, I'd like to get feedback on whether folks see this as a worthwhile thing to have...


> test harness usage of useprocess needs improvement
> --------------------------------------------------
>
>          Key: DERBY-1141
>          URL: http://issues.apache.org/jira/browse/DERBY-1141
>      Project: Derby
>         Type: Improvement

>   Components: Test
>     Reporter: Myrna van Lunteren
>     Assignee: Myrna van Lunteren
>  Attachments: DERBY-1141_20060413.diff, DERBY-1141_20060413.stat
>
> The 'old' functionTests harness has property useprocess, which, when set to false, causes tests within a suite to run from within one jvm, thus creating databases and files in the same directory.
> This functionality is useful for debugging test runs within an IDE.
> Also, it is potentially useful for speeding up testruns.
> However, currently, there are some problems and shortcomings of this functionality.
> - For instance, when running a networkserver test with framework DerbyNet of type .java one sees the message 'Security Manager not installed' show up within the test output, thus causing the test to fail.
> - running derbynet(client)mats suite with useprocess=false doesn't actually make things any faster,  because networkserver gets started fresh for every test. 
>   However, it should be feasible to start networkserver with the first test in a suite, then use that 
>   networkserver  for subsequent tests and shutdown networkserver when the suite finishes.
> - Also, (reported by Mike Matrigali off-list) currently, the following:
> ------------------
> java -Duseprocess=false -Doutputdir=<snippeddirname>/newout  -Dkeepfiles=true org.apache.derbyTesting.functionTests.harness.RunTest lang/closed.java
> ------------------
> makes the test fall out of the harness with: Exception in thread "main"
> and in the .tmp:
> ------------------
> java.security.AccessControlException: access denied (java.io.FilePermission <snippeddirname>\newout read)
>         at java.security.AccessControlContext.checkPermission(AccessControlContext.java:269)
>         at java.security.AccessController.checkPermission(AccessController.java:401)
>         at java.lang.SecurityManager.checkPermission(SecurityManager.java:524)
>         at java.lang.SecurityManager.checkRead(SecurityManager.java:863)
>         at java.io.File.exists(File.java:678)
>         at java.io.Win32FileSystem.canonicalize(Win32FileSystem.java:360)
>         at java.io.File.getCanonicalPath(File.java:513)
>         at org.apache.derbyTesting.functionTests.harness.RunTest.execTestNoProcess(RunTest.java:2370)
>         at org.apache.derbyTesting.functionTests.harness.RunTest.testRun(RunTest.java:443)
>         at org.apache.derbyTesting.functionTests.harness.RunTest.main(RunTest.java:302)
> --------------------------------
>   removing the -Doutputdir gets around that problem, but shouldn't be needed.
> - there is the reference http://www.nabble.com/AccessControlException-when-running-functional-tests-t1321374.html#a3526947
>   I couldn't duplicate this yet, maybe more info to follow.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira