Mailing-List: contact derby-dev-help@db.apache.org; run by ezmlm
Precedence: bulk
Reply-To: <derby-dev@db.apache.org>
Message-ID: <928769767.1141408539557.JavaMail.jira@ajax.apache.org>
Date: Fri, 3 Mar 2006 18:55:39 +0100 (CET)
From: "Mike Matrigali (JIRA)" <derby-dev@db.apache.org>
To: derby-dev@db.apache.org
Subject: [jira] Created: (DERBY-1074) enhance test harness to allow control
 of running test by JVM id, rather than just JVM version
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

enhance test harness to allow control of running test by JVM id, rather than just JVM version
---------------------------------------------------------------------------------------------

         Key: DERBY-1074
         URL: http://issues.apache.org/jira/browse/DERBY-1074
     Project: Derby
        Type: Improvement
  Components: Test  
    Versions: 10.1.2.2    
    Reporter: Mike Matrigali


In looking at the growing list of issues in the regression test component, it looks like we are seeing a number of 
issues that are specific to a particular JVM on a particular JVM version.  The harness currently only allows control of
running a test either in the a suite, or not in a particular jvm version.   It seems like we have seen a need to be
able to disable a test in a particular jvm version (ie. 1.4.2) on a particular jmv (ie. ibm version) on a particular OS (ie linux version xxx).

I believe the model we are trying to get to is that we try to reach 0 issues in the regression test suite, ie. there are no 
"expected" errors.  There are always going to be bugs that don't have anyone who is interested in fixing them, and those
should be reported in JIRA.  At that point if they are not going to be fixed, they should be removed from the particular 
environment they break in, there is no point in rerunning a breaking test which already has been diagnosed and reported.

There are currently issues being left in, because the test harness does not have the correct granularity.  Removing the test
would require it to be not run in some environments where it does pass and thus we would lose some code coverage.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira