db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rick Hillegas <Richard.Hille...@Sun.COM>
Subject Re: Question about client compatibility testing and DERBY-516 (paging Rick #:))
Date Mon, 07 Nov 2005 19:03:12 GMT
Hi Kathey,

My initial ramblings on this topic start out at the end of August in the 
email thread "client/server compatibility testing". There I worried that 
over time, the compatibility tests could grow large (taking maybe 5 
minutes per combination) and so, if run for all the combinations, would 
make derbyall take too long. That's why we're only runing one 
combination as part of derbyall. That combination doesn't really track a 
compatibility issue, it just tracks regressions which might creep into 
the test as other code changes.

I'm certainly in favor of running all the combinations on a nightly or 
weekly basis and as a sanity check when cutting release candidates.

I like the idea of wiring some upgrade tests into derbyall. I think it 
would tend to enforce the discipline that developers write upgrade logic 
and tests at the same time that they checkin code which raises upgrade 
problems. Writing upgrade tests is a tricky problem. Technically, an 
upgrade test case consists of two pieces: 1) the Setup code which is 
compiled and run against the previous release and 2) the Verification 
code which is compiled and run against the current release. If you want 
to tackle upgrade testing by checking in jar files, you'll have to add 
some tricky build logic which builds Setup code against the previous 
release and Verification code against the current release.

Just for comparison, here's the equally tricky solution which we adopted 
for Cloudscape back in the old days: 1) The Setup code was checked into 
the previous release branch, 2) the test developer ran a script to 
populate a previous rev database with all the Setup cases then tar up 
the resulting database, 3) that tarball was checked into the current 
release, 4) the derbyall piece of the upgrade tests then grabbed that 
tarball, upgraded it, and ran the Verification code against it.


Kathey Marsden wrote:

>I noticed that DERBY-516 was closed with the addition of  the new test
>jdbcapi\CompatibilityTest.java. which adds a nice framework for some
>basic compatibility testing.
>By default it seems that this test  just runs with 10.2 client and 10.2
>server so doesn't give us any additional coverage in derbyAll for
>compatibility testing.  I was wondering if there were plans to check the
>derby release jars into the trunk so that the compatibility testing
>would be performed as part of the normal testing procedures?  These jars
>could also be used in  DERBY-514 to  enable upgrade testing.

View raw message