db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kristian Waagan <Kristian.Waa...@Sun.COM>
Subject Re: [jira] Commented: (DERBY-1141) test harness usage of useprocess needs improvement
Date Wed, 19 Apr 2006 11:27:45 GMT
Andreas Korneliussen wrote:
> Kristian Waagan wrote:
>> Myrna van Lunteren wrote:
>>> On 4/18/06, *John H. Embretsen (JIRA)* <derby-dev@db.apache.org 
>>> <mailto:derby-dev@db.apache.org>> wrote:
>>
>> [snip - details and useprocess=false discussion]
>>
>>>
>>>     JUnit and useprocess=false:
>>>     ----------------------------------------------------
>>>
>>>     Some may be surprised to find that junit tests are run in a separate
>>>     process by default (instead of being skipped) when useprocess=false.
>>>     However, I think I see some advantages of doing it this way, so I
>>>     accept (a bit reluctantly, perhaps) this behavior for now.
>>>
>>>     In the patch I also find:
>>>
>>>     -        if ( useprocess )
>>>     +           // Always create a process if testType is junit,
>>>     otherwise testRunner
>>>     +           // would exit a suite prematurely, and we can't debug
>>>     that anyway and
>>>     +           // they should run quick anyway.
>>>     +        if ( useprocess || (!useprocess && 
>>> testType.equals("junit")))
>>>
>>>     I'm not sure I understand/remember exactly why junit tests cannot be
>>>     run with useprocess=false ( i.e. why "testRunner" (I assume this
>>>     means junit.*ui.TestRunner) would exit a suite prematurely). Feel
>>>     free to educate me on this subject (otherwise I guess I'll look into
>>>     it myself when time permits) ;)
>>>
>>>  
>>>  
>>> Yes, I was referring to junit.*ui.TestRunner. I wanted to enable as 
>>> many test as possible when running useprocess=false, and I thought 
>>> skipping the junit test would be an increasingly bad idea as we'd get 
>>> more and more junit tests...
>>>  
>>> So I experimented running the junit test as a separate process (I 
>>> just copied how they're run when useprocess=true (the default)), the 
>>> first junit test ran successfully and that was all that ran, the 
>>> entore RunSuite process just stopped. I assumed that TestRunner must 
>>> be doing a System.exit() somewhere. Maybe I was off with that 
>>> conclusion/assumption...
>>
>> I think you are correct with your conclusion/assumption. A quick grep 
>> of the JUnit source code (3.8.1), shows matches for System.exit in all 
>> of textui.TestRunner, swingui.TestRunner and awtui.TestRunner.
>>
>> Since we would want a text/console based test runner, we may have to 
>> write our own if we want to run JUnit tests with useprocess=false.
>> When looking at the source of textui.TestRunner, I don't think this 
>> would be a very different task. It might even be possible to do a 
>> quick hack, but I do not know if this is allowed according to the 
>> license (CPL) or if we want to do this.
>>
>> Can anyone with knowledge of CPL (Common Public License - v 1.0) shed 
>> some light on what we can and can't do?
>>
> To make a new TestRunner, you may i.e make a new class which inherits 
> from junit.runner.BaseTestRunner.
> 
> In terms of legal, I do not see how this would be different from making 
> a class which inherits from junit.framework.TestCase, which we already 
> do with junit tests. However I am not sure what exactly you mean by 
> making a quick hack  - did you mean modifying the existing 
> junit.runner.TestRunner class?

Yes, primarily to see how/if things work, before we spend time and 
resources on writing our own TestRunner. But as I said, writing our own 
runner should be pretty easy.

> 
> A new TestRunner should imo. do the following:
> * when starting a Test, it should print that it starts a test, instead 
> of printing .
> * when a test fails, it should immediately print the failure, instead of 
>  waiting untilt the test completes.

In addition to the above, it should also print a summary with the number 
of tests, number of failures and errors (as the textui.TestRunner does). 
The output might require some thought, depending on how the tests are 
run (ie. a single run of one big suite, or multiple runs of smaller suites).

> 
> As for integration with the harness, the messages printed from junit 
> tests / TestRunners, should be prefixed with something which can easily 
> be filtered by Sed - to avoid making diff files for junit tests.

I totally agree. Introducing diff files for JUnit tests would be a huge 
regression in my opinion.



--
Kristian

> 
> 
> Andreas
> 


Mime
View raw message