db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mike Matrigali (JIRA)" <derby-...@db.apache.org>
Subject [jira] Created: (DERBY-1074) enhance test harness to allow control of running test by JVM id, rather than just JVM version
Date Fri, 03 Mar 2006 17:55:39 GMT
enhance test harness to allow control of running test by JVM id, rather than just JVM version
---------------------------------------------------------------------------------------------

         Key: DERBY-1074
         URL: http://issues.apache.org/jira/browse/DERBY-1074
     Project: Derby
        Type: Improvement
  Components: Test  
    Versions: 10.1.2.2    
    Reporter: Mike Matrigali


In looking at the growing list of issues in the regression test component, it looks like we
are seeing a number of 
issues that are specific to a particular JVM on a particular JVM version.  The harness currently
only allows control of
running a test either in the a suite, or not in a particular jvm version.   It seems like
we have seen a need to be
able to disable a test in a particular jvm version (ie. 1.4.2) on a particular jmv (ie. ibm
version) on a particular OS (ie linux version xxx).

I believe the model we are trying to get to is that we try to reach 0 issues in the regression
test suite, ie. there are no 
"expected" errors.  There are always going to be bugs that don't have anyone who is interested
in fixing them, and those
should be reported in JIRA.  At that point if they are not going to be fixed, they should
be removed from the particular 
environment they break in, there is no point in rerunning a breaking test which already has
been diagnosed and reported.

There are currently issues being left in, because the test harness does not have the correct
granularity.  Removing the test
would require it to be not run in some environments where it does pass and thus we would lose
some code coverage.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira


Mime
View raw message