Mailing-List: contact derby-dev-help@db.apache.org; run by ezmlm
Precedence: bulk
Reply-To: <derby-dev@db.apache.org>
Received-SPF: neutral (athena.apache.org: local policy)
Message-ID: <515F05DC.4090604@sbcglobal.net>
Date: Fri, 05 Apr 2013 10:11:56 -0700
From: Katherine Marsden <kmarsdenderby@sbcglobal.net>
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64;
 rv:17.0) Gecko/20130307 Thunderbird/17.0.4
MIME-Version: 1.0
To: derby-dev@db.apache.org
Subject: 10.10 upgrade and compatibility testing
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

I filed https://issues.apache.org/jira/browse/DERBY-6152 for the 10.10 
upgrade and compatibility testing and started doing some work on it but 
I think this is going to be more helpful for 10.10.2.1 by the time I 
finish. The major problems are:

Just running the 10.8 tests against the 10.10 product jars fails 
miserably  ( Failures: 433,  Errors: 74)
The failures fall into three categories.

1) Cascade failures and cleanup problems.
      I hope to cleanup the tests and test infrastructure to cleanup 
properly so that tests cannot interfere with the ones ahead.  I think 
fixtures and tests that cleanup after themselves  should be a general 
strategy when adding new tests.

2) Tests in the functional tests for internal or unstable interfaces 
that have now changed behavior with 10.10.
     It would be nice in the future to have a way to flag this testing 
so it can be avoided on a compatibility test run. I am not sure the best 
way to do that, but welcome ideas, maybe a system property and then a if 
(testInternal()) block around the test code would allow us to run 
compatibility tests.

3) Real behavior changes that might require release notes or be a problem.
    These are the ones we want to see

Anyway, I will forge ahead with the cleanup issues which will be I hope 
generally useful and welcome input on how to make this testing easier in 
the future.

Best

Kathey