Mailing-List: contact derby-dev-help@db.apache.org; run by ezmlm
Precedence: bulk
Reply-To: <derby-dev@db.apache.org>
Received-SPF: pass (asf.osuosl.org: local policy)
Date: Fri, 18 Nov 2005 16:29:37 +0100
From: John Embretsen <John.Embretsen@Sun.COM>
Subject: Re: Test Harness useprocess=false question
In-reply-to: <c25576af0511171528hd489940u9862bf4456c95caf@mail.gmail.com>
To: derby-dev@db.apache.org
Message-id: <437DF361.3090402@sun.com>
Organization: Sun Microsystems (Trondheim, Norway)
MIME-version: 1.0
Content-type: multipart/mixed; boundary="Boundary_(ID_lccUfnXibloD1U1BlXT5EQ)"
User-Agent: Mozilla/5.0 (X11; U; SunOS i86pc; en-US; rv:1.7) Gecko/20041221
References: <437CFEEE.90208@debrunners.com>
 <c25576af0511171528hd489940u9862bf4456c95caf@mail.gmail.com>

This is a multi-part message in MIME format.

--Boundary_(ID_lccUfnXibloD1U1BlXT5EQ)
Content-type: text/plain; charset=us-ascii; format=flowed
Content-transfer-encoding: 7BIT

Myrna van Lunteren wrote:
> On 11/17/05, *Daniel John Debrunner* <djd@debrunners.com 
> <mailto:djd@debrunners.com>> wrote:
> 
>     The test harness can run tests within the same jvm if useprocess=false.
>     This is set for the nist suite, so that all the sub-scripts are run
>     in a
>     single JVM, mainly for performance reasons.
> 

[snip]

>  
> This also implies that any tests that need special flags cannot be run 
> with useprocess = false.

Are there other cases (other than the nist tests) where we should be 
able to run with useprocess set to false?

I dug into some of the test harness code a while ago, trying to 
understand a little better how the machinery works (I won't say anything 
about how successful I was... ;) ). I think I found other issues with 
useprocess=false...

For example, there are a number of tests that include a call to 
System.exit(), which means that when useprocess=false, the test shuts 
down the JVM, and the harness is not able to process the results and 
clean up after itself.

README.htm in the java/testing directory states the following:

<Quote>

useprocess

(default=true) Controls whether RunTest runs the test in a separate VM
or in a thread in harness VM. It is potentially useful for debugging
tests. Unit tests are not (yet) runnable with "useprocess=false", though.

<end Quote>

It does not say anything about which tests "useprocess=false" may or may 
not work with, other than unit tests.

So I guess the question is: Is this a bug? If so, is it a documentation 
bug or a bug in each of the tests listed in the attachment to this e-mail?


-- 
John

--Boundary_(ID_lccUfnXibloD1U1BlXT5EQ)
Content-type: text/plain; name=SystemExit-tests.txt
Content-transfer-encoding: 7BIT
Content-disposition: inline; filename=SystemExit-tests.txt

Executing: find ../clean/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/ -print | xargs grep -n 'System.exit(' | grep -v '.svn'

../clean/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/lang/SpillHash.java:104:            System.exit(0);
../clean/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/lang/SpillHash.java:109:            System.exit(1);
../clean/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/lang/timestampArith.java:312:            System.exit(1);
../clean/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/lang/timestampArith.java:358:            System.exit(1);
../clean/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/lang/timestampArith.java:364:            System.exit(1);
../clean/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/lang/timestampArith.java:366:        System.exit(0);
../clean/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/lang/CharUTF8.java:118:                 System.exit(-1);
../clean/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/lang/longStringColumn.java:118:                 System.exit(-1);
../clean/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/derbynet/testSecMec.java:81:                    System.exit(-1);
../clean/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/derbynet/DerbyNetAutoStart.java:98:            System.exit(0);
../clean/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/derbynet/DerbyNetAutoStart.java:103:            System.exit(1);
../clean/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/derbynet/DerbyNetAutoStart.java:149:                    System.exit(0);
../clean/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/derbynet/DerbyNetAutoStart.java:171:            System.exit(1);
../clean/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/derbynet/DerbyNetAutoStart.java:246:            System.exit(1);
../clean/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/derbynet/dataSourcePermissions_net.java:87:                     System.exit(-1);
../clean/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/store/TestDiskHashtable.java:93:        System.exit( (failed == 0) ? 0 : 1);

--Boundary_(ID_lccUfnXibloD1U1BlXT5EQ)--