db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andreas Korneliussen <Andreas.Kornelius...@Sun.COM>
Subject Re: Features of the JUnit test execution harness
Date Fri, 03 Feb 2006 12:05:06 GMT
Myrna van Lunteren wrote:
> On 2/2/06, *Andreas Korneliussen* <Andreas.Korneliussen@sun.com 
> <mailto:Andreas.Korneliussen@sun.com>> wrote:
>     I think the work currently done on DERBY-874 was mainly to improve the
>     DerbyJUnitTest's JavaDoc, and to log exceptions. So I would not throw
>     that away.
>     However I do propose to change DerbyJUnitTest to move out everything
>     about configuration into a separate class.
> cool. thx for the reply.
> I now noticed that the wiki says all suggestions are to be put on the 
> list, so here I go rather than plopping them directly on the wiki:
Great feature list.

What I think is important is that we provide a common mechanism to get 
access to the configuration of the test.

If someone then need to do some framework/version specific logic, they 
have a mechanism to get i.e the framework enum, and can write logic 
based on that.

> I think the following could qualify as 'more details' to the jvm, 
> framework, version specific logic:
> 1. jvm-specific:
> 1.1.
> not all parameters are consistent for all jvms. Think here of jit 
> settings / configurations, memory settings. For j2ME testing, that jvm 
> doesn't come with a DriverManager implementation, so already from the 
> start you know you have to go with DataSources.

So I guess what you are saying is that if the test framework provides a 
common mechanism to give a Connection to a derby database, it should go 
through a DataSource, instead of using DriverManager ?

> 1.2. Different versions of a vendor's jvm also may have slightly 
> different implementations resulting in slightly different behavior - 
> e.g. of the order of rows, for instance, or rounding of certain numeric 
> values.
> 1.3. Some behavior will be only supported by later versions...
> 2. version specific.
> This really falls back to the discussion on ...(can't find right now, 
> raman's working on it, I think)... re mixed version testing. I think the 
> conclusion was that the harness needs a way to cope with results from 
> newer servers and clients - if they differ from results with same 
> versions as the harness.
> 3. framework specific
> The tests needs to be able to cope with the following
> 3.1. different client drivers (e.g. DerbyClient, IBM Universal JDBC Driver)
> 3.2. server may need to be started by harness, or not
> 3.3. server may be on the localhost, or be running on a remote machine.
>          certain individual tests may not be able to run in with this 
> mechanism...
> 3.4 should be able to have the harness start the server in a differrent jvm.
> 4. one thing the current harness has no way of doing is to cope with 
> different OSs. For instance, sometimes there are slight differences in 
> behaviour of the same jvm version on different OSs. Like slightly 
> different error messages (although this may be irrelevant if we're not 
> gathering & comparing output).
> I think the following details would be useful (in addition to the above 
> and item 1 on the wiki):
> - there must be a way to skip individual tests without causing an error 
> but with an informational message for certain configurations. eg. 
> absence of optional jars (specifically thinking of db2jcc.jar), 
> unsupported functionality with older jvms..., or when there is a problem 
> that's being worked on, or that's been referred to some other 
> organization ( e.g. in the case of jvm bugs, OS bugs...).
> - some way to compare runtimestatistics.
>    Currently this is done by comparing the output, I have a hard time 
> thinking of another mechanism.

I am not sure which runtimestatistics you think of. Which output ? 
Output from a SQL statement ?



View raw message