db-derby-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Db-derby Wiki] Update of "HandlingKnownFailures" by BryanPendleton
Date Wed, 02 Aug 2006 15:56:43 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Db-derby Wiki" for change notification.

The following page has been changed by BryanPendleton:

The comment on the change is:
Some useful suggestions from Mike.

New page:
At times, you will find yourself confronted with the situation in which a certain test is
failing, and will be failing for some period of time. For example, there may be a bug, which
has been logged in Jira and which is well understood, but which won't be fixed today, or tomorrow.

In general, the development community tries not to leave "expected" failures in the head of
the tree. Our goal is that if you check out the head of the trunk, and build it and run the
tests, they will all pass.

Here are some possible techniques you can use if you find yourself in this situation:

 * Many tests are run as part of multiple suites. If a test is failing only in one suite,
you can remove the test from that suite and leave it in the others, with comments indicating
to restore the test to the suite when the bug is fixed.
 * Depending on the test, you may be able to remove a very small part of the test, perhaps
commenting out just a single statement of the test code, until the bug is fixed.
 * At other times with the canon-based tests we have added comments to the test output itself
saying something like "the next few lines are wrong due to DERBY-xxx"

In general, please don't just update the master file so that it contains erroneous results
without at least putting comments into both the test and the test output indicating that the
output is known to be wrong, and is due to bug DERBY-xxx.

That has the benefit of making it very clear when the bug is fixed and a new diff appears,
why that is happening.

View raw message