db-ojb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From oliver.m...@ppi.de
Subject non-deterministic test failures, was: RE: junit test failures
Date Mon, 10 Nov 2003 16:07:37 GMT
Hello again,

I am a little bit annoyed at the moment because
I spent several hours figuring out how I corrupted
test cases until I finally discovered that 
some test failures are not reproducable.

More precisely:
I checked out current CVS head into an empty 
directory and then repeated

bin\build.bat junit-no-compile junit-report

several times.  In between I saved the 
target/test directory. Then I compared the 
results.  There are various results.  Some
test fail non-deterministically.  Besides, there
seem to be dependencies such as:

test Y succeeds iff test X runs before test Y.

Discovering that took quite a while, and I was
only doing the tests to make sure that some minor
cleanup has not done any harm.

I wonder if someone else has experienced that?
How often do you run the tests?  

In order to benefit from the junit tests, every 
developer should be able to rely on that all tests 
that are not marked as 'allowed to fail' do not fail
unless he or she corrupts them.

> -----Original Message-----
> From: oliver.matz@ppi.de [mailto:oliver.matz@ppi.de]
> Sent: Monday, November 10, 2003 3:03 PM
> To: ojb-dev@db.apache.org

> > Can I compare these results to the 'official' ones?
> > The link http://db.apache.org/ojb/junit-report.html
> > on http://db.apache.org/ojb/maven-reports.html is broken.
> the tests have changed, but this issue is still open.
> Does someone take care of this?
> Can we achieve any consensus about how to deal with failing
> tests?  I have stated my opinion about this in another
> thread.

This issue is really important to me.

I will have a look at some tests that fail sometimes.


To unsubscribe, e-mail: ojb-dev-unsubscribe@db.apache.org
For additional commands, e-mail: ojb-dev-help@db.apache.org

View raw message