db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel John Debrunner <...@apache.org>
Subject Re: JUnit disagnostice (Was Re: Some JUnit tests now split out from old harness and do not run as part of derbyall)
Date Thu, 19 Oct 2006 22:47:49 GMT
Øystein Grøvlen wrote:
> Vemund Ostgaard wrote:
> 
>> When running the top-level suite now, the only output I got was a lot 
>> of dots, the time it took to run the suite and the number of OK tests 
>> run.
>>
>> Have anyone considered a framework or interface for logging events and 
>> information in the junit tests? I think it will be very difficult in 
>> the future to analyse test failures only based on the text from an 
>> assertion failure or exception. Especially when we start porting 
>> multhreaded stress tests and other more complicated parts of the old 
>> suite.
> 
> I totally agree with you.  I had an experience when porting 
> LobLengthTest to junit that illustrate this.  When I had the test to 
> jdbcapi/_Suite, OOM errors occurred and the test hanged when I ran it 
> with the DerbyNetClient.  There was no practical way to find out which 
> test was hanging.  I could have counted dots and tried to determine 
> which test case that represented, but that would have been both error 
> prone and very time consuming. (The problem went away when Fernanda 
> checked in the clean-up to the SUR-test.)
> 
>>
>> It would also give a warm fussy feeling to be able to see that the 
>> tests actually did something (in addition to all the dots written by 
>> the TestRunner).
> 
> I agree on this too.  The way it is today, if I by mistake disable some 
> test, this is very difficult to detect.

The Swing test runner shows more information about which tests have run 
and passed. Remember the textui test runner is just a simple test runner.

I would encourage investigation of what others have done in this area, 
see if there are better test runners, rather than invent a new 
mechanism. I would like that the Derby JUnit tests continue to run as 
other Junit tests so others can run & integrate them up easily. If the 
default running of Derby's JUnit tests produce a lot of output that will 
confuse others who are used to the model where a test produces no output 
if it succeeds.

Dan.


Mime
View raw message