db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Embretsen <John.Embret...@Sun.COM>
Subject Re: Test Harness useprocess=false question
Date Sun, 20 Nov 2005 20:45:33 GMT
Friday, November 18, 2005, 8:31:28 PM, Myrna van Lunteren wrote:

>> On 11/18/05, John Embretsen <John.Embretsen@sun.com> wrote:

>> Are there other cases (other than the nist tests) where we should be
>> able to run with useprocess set to false?
>  
>
> To be frank, a former Cloudscape QA had at one point made it his goal
> to clean up and make as many tests as possible run with
> useprocess=false. However, that person was sent packing somewhere in
> the take-overs and lay-off rounds (first by Informix, then by IBM)
> before finishing that project. And it has not been high on anyones list since.

I see, that's interesting...

> I think most tests *ought* to be able to run with useprocess set to
> false. However, this requires a certain mindset of the test writer -
> as it involves leaving the database behind exactly as you found it; or
> if absolutely required, use identifiers that are not used anywhere
> else in the tests. This is not always the case - we happily rely on
> the test harness cleaning up after us, and it's hard to think of
> imaginative name for a test table with an integer and character column.
>
> As I need to use useprocess for DERBY-413, I am cleaning up the tests
> in derbynetclientmats (which includes jdbcapi, jdbc20 and jdk14 suites), but I'm not
going beyond.

OK, so are you running into InstantiationExceptions anywhere? I noticed
that with useprocess=false, the main method of the test class is invoked
through reflection. RunTest.java calls RunClass.java, which does this by

mainMethod.invoke(testClass.newInstance(), arguments);

Now, the call to newInstance() will fail if the test class does not
include a (public) nullary constructor, or because of some other issue
(see JavaDoc for java.lang.Class). A simple solution to this problem (I
know that some java tests does not include such a constructor) would be
to pass "null" instead of "testClass.newInstance()" to the invoke
method, since the method called (main) is static anyway... ("If the
underlying method is static, then the specified obj argument is ignored.
It may be null.", from JavaDoc for java.lang.reflect.Method.invoke(...)).

> So, in short, as you found the only tests that are currently running with useprocess=false
are nist.
>
>  
>> I dug into some of the test harness code a while ago, trying to
>> understand a little better how the machinery works (I won't say anything
>> about how successful I was... ;) ).
>
> :-) well, it works after a fashion...sort of. The whole thing could
> do with a clean-up, but it seems more sensible to spend time on
> writing JUnit tests, at this point.

I could not agree more... I liked Dan's remark about this not so long ago
(something like: Trying to fix the current harness instead of
starting afresh with JUnit tests would be insane) ;)

>>  I think I found other issues with
>> useprocess=false...
>  
>  
>  
> Very likely.

Another one is described above...

>> For example, there are a number of tests that include a call to
>> System.exit(), which means that when useprocess=false, the test shuts
>> down the JVM, and the harness is not able to process the results and
>> clean up after itself.

>> So I guess the question is: Is this a bug? If so, is it a documentation
>> bug or a bug in each of the tests listed in the attachment to this e-mail?

> Not a bug perse, but if someone wants, they can open a jira
> enhancement & start making the rest of the tests (not including
> derbynetmats, derbynetclientmats, jdbcapi, jdbc20, jdk14 and multi, 'cause I'm doing
those) run ok.

Sounds good. (Although I will not be that someone anytime soon...)

-- 
John


Mime
View raw message