db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kathey Marsden (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (DERBY-6210) Create a mechanism to exclude some testing of internal interfaces and likely to change behavior from compatibility testing
Date Wed, 15 May 2013 20:49:16 GMT

     [ https://issues.apache.org/jira/browse/DERBY-6210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Kathey Marsden updated DERBY-6210:
----------------------------------

    Issue Type: Improvement  (was: Bug)
    
> Create a mechanism to exclude some testing of internal interfaces and likely to change
behavior from compatibility testing
> --------------------------------------------------------------------------------------------------------------------------
>
>                 Key: DERBY-6210
>                 URL: https://issues.apache.org/jira/browse/DERBY-6210
>             Project: Derby
>          Issue Type: Improvement
>            Reporter: Kathey Marsden
>
> For compatibility and upgrade testing, in order to expose bugs and omitted release notes
I run the tests from an older branch against new releases.  The premise is that the functional
tests are really just a JDBC application which should continue to pass with the new version.
> The JUnit tests for Derby though are not pure functional tests which are backward compatible.
 Often they make checks on things that are likely to change with new versions. This creates
a large amount of noise in the testing: For example:
> 1) Checks for the exact contents or number of system tables.
> 2) Checks for "Not implemented" JDBC API calls which might become implemented.
> 3) Tests for client specific behavior which we expect might change to match embedded.
> 4) Unit tests which use internal interfaces that are likely to change.
> 5) Metadata tests which test the exact number of columns when 
> 6) Diff based tests which are likely to change with message changes.
> It would be good to have a flag to omit these types of tests when doing compatibility
testing.
> My thought is to have a system property derby.tests.testCompat=true   and a method in
BaseTestCase to test the property testCompat().
> Then blocks of testing which should not be run or should have a different behavior with
compatibility testing can be flagged as:
> if (! testCompat())  {
> // do testing  that we think might change, e.g. system table query.
> }
> This will allow the more detailed testing under normal circumstances but take some of
the pain out of the compatibility testing moving forward.  It would require folks to think
as they add new tests whether their testing as to whether they expect the tested behavior
 to remain stable moving forward and put the block around it if they do not.
> I welcome feedback on whether this or some other approach is preferable.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message