db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Myrna van Lunteren <m.v.lunte...@gmail.com>
Subject Re: Features of the JUnit test execution harness
Date Fri, 03 Feb 2006 01:22:19 GMT
On 2/2/06, Andreas Korneliussen <Andreas.Korneliussen@sun.com > wrote:
>
> I think the work currently done on DERBY-874 was mainly to improve the
> DerbyJUnitTest's JavaDoc, and to log exceptions. So I would not throw
> that away.
>
> However I do propose to change DerbyJUnitTest to move out everything
> about configuration into a separate class.


cool. thx for the reply.

I now noticed that the wiki says all suggestions are to be put on the list,
so here I go rather than plopping them directly on the wiki:

I think the following could qualify as 'more details' to the jvm, framework,
version specific logic:

1. jvm-specific:
1.1.
not all parameters are consistent for all jvms. Think here of jit settings /
configurations, memory settings. For j2ME testing, that jvm doesn't come
with a DriverManager implementation, so already from the start you know you
have to go with DataSources.
1.2. Different versions of a vendor's jvm also may have slightly different
implementations resulting in slightly different behavior - e.g. of the order
of rows, for instance, or rounding of certain numeric values.
1.3. Some behavior will be only supported by later versions...

2. version specific.
This really falls back to the discussion on ...(can't find right now,
raman's working on it, I think)... re mixed version testing. I think the
conclusion was that the harness needs a way to cope with results from newer
servers and clients - if they differ from results with same versions as the
harness.

3. framework specific
The tests needs to be able to cope with the following
3.1. different client drivers (e.g. DerbyClient, IBM Universal JDBC Driver)
3.2. server may need to be started by harness, or not
3.3. server may be on the localhost, or be running on a remote machine.
         certain individual tests may not be able to run in with this
mechanism...
3.4 should be able to have the harness start the server in a differrent jvm.

4. one thing the current harness has no way of doing is to cope with
different OSs. For instance, sometimes there are slight differences in
behaviour of the same jvm version on different OSs. Like slightly different
error messages (although this may be irrelevant if we're not gathering &
comparing output).

I think the following details would be useful (in addition to the above and
item 1 on the wiki):
- there must be a way to skip individual tests without causing an error but
with an informational message for certain configurations. eg. absence of
optional jars (specifically thinking of db2jcc.jar), unsupported
functionality with older jvms..., or when there is a problem that's being
worked on, or that's been referred to some other organization (e.g. in the
case of jvm bugs, OS bugs...).

- some way to compare runtimestatistics.
   Currently this is done by comparing the output, I have a hard time
thinking of another mechanism.

Ok, that'll do for now...:-)

Myrna

Mime
View raw message