db-ojb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jakob Braeuchi <jbraeu...@gmx.ch>
Subject Re: non-deterministic test failures, was: RE: junit test failures
Date Wed, 12 Nov 2003 19:43:28 GMT


Armin Waibel wrote:

> Hi,
> 
> Jakob Braeuchi wrote:
> 
>> hi oli,
>>
>> the sequence of the testcases is sometimes a problem. the querytest 
>> ie. relies on a certain number of rows to be available in the database 
>> that's why it may fail if not run as first testcase.
> 
> 
> I don't know these tests in detail, but I think this should be
> fixed if possible. Test shouldn't rely on other tests.

the do not rely on each other. the problem is that some testcases modify 
the database and do not undo the changes they have made.

> 
>> also some testcase fail if run more than once without reinitializing 
>> the database.
> 

same problem .

> 
> This should be avoided too, because it is not a "real life test"
> only run the api against empty "virgin" database tables
> (e.g. maybe some side-effects only occur when re-run tests against 
> filled tables).
> 
> just my 2 cent ;-)
euro cents ?
> 
> regards,
> Armin

jakob

> 
>> i run alltests before commiting any change to the repository.
>>
>> jakob
>>
>>
>> oliver.matz@ppi.de wrote:
>>
>>> Hello again,
>>>
>>> I am a little bit annoyed at the moment because
>>> I spent several hours figuring out how I corrupted
>>> test cases until I finally discovered that some test failures are not 
>>> reproducable.
>>>
>>> More precisely:
>>> I checked out current CVS head into an empty directory and then repeated
>>>
>>> bin\build.bat junit-no-compile junit-report
>>>
>>> several times.  In between I saved the target/test directory. Then I 
>>> compared the results.  There are various results.  Some
>>> test fail non-deterministically.  Besides, there
>>> seem to be dependencies such as:
>>>
>>> test Y succeeds iff test X runs before test Y.
>>>
>>> Discovering that took quite a while, and I was
>>> only doing the tests to make sure that some minor
>>> cleanup has not done any harm.
>>>
>>> I wonder if someone else has experienced that?
>>> How often do you run the tests? In order to benefit from the junit 
>>> tests, every developer should be able to rely on that all tests that 
>>> are not marked as 'allowed to fail' do not fail
>>> unless he or she corrupts them.
>>>
>>>
>>>> -----Original Message-----
>>>> From: oliver.matz@ppi.de [mailto:oliver.matz@ppi.de]
>>>> Sent: Monday, November 10, 2003 3:03 PM
>>>> To: ojb-dev@db.apache.org
>>>
>>>
>>>
>>>
>>>>> Can I compare these results to the 'official' ones?
>>>>> The link http://db.apache.org/ojb/junit-report.html
>>>>> on http://db.apache.org/ojb/maven-reports.html is broken.
>>>>
>>>>
>>>>
>>>> the tests have changed, but this issue is still open.
>>>>
>>>> Does someone take care of this?
>>>>
>>>> Can we achieve any consensus about how to deal with failing
>>>> tests?  I have stated my opinion about this in another
>>>> thread.
>>>
>>>
>>>
>>>
>>> This issue is really important to me.
>>>
>>> I will have a look at some tests that fail sometimes.
>>>
>>> Olli
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: ojb-dev-unsubscribe@db.apache.org
>>> For additional commands, e-mail: ojb-dev-help@db.apache.org
>>>
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: ojb-dev-unsubscribe@db.apache.org
>> For additional commands, e-mail: ojb-dev-help@db.apache.org
>>
>>
>>
> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: ojb-dev-unsubscribe@db.apache.org
> For additional commands, e-mail: ojb-dev-help@db.apache.org
> 
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: ojb-dev-unsubscribe@db.apache.org
For additional commands, e-mail: ojb-dev-help@db.apache.org


Mime
View raw message