db-ojb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Armin Waibel <ar...@code-au-lait.de>
Subject Re: non-deterministic test failures, was: RE: junit test failures
Date Wed, 12 Nov 2003 19:50:40 GMT
Hi Jakob,

Jakob Braeuchi wrote:
> 
> 
> Armin Waibel wrote:
> 
>> Hi,
>>
>> Jakob Braeuchi wrote:
>>
>>> hi oli,
>>>
>>> the sequence of the testcases is sometimes a problem. the querytest 
>>> ie. relies on a certain number of rows to be available in the 
>>> database that's why it may fail if not run as first testcase.
>>
>>
>>
>> I don't know these tests in detail, but I think this should be
>> fixed if possible. Test shouldn't rely on other tests.
> 
> 
> the do not rely on each other. the problem is that some testcases modify 
> the database and do not undo the changes they have made.
> 
ahh, ok that's true. I think 90% of the test cases do not
undo work. Maybe it is possible to make the tests more
independable. E.g. use a 'flag' in objects name to find
the according object.

regards,
Armin

>>
>>> also some testcase fail if run more than once without reinitializing 
>>> the database.
>>
>>
> 
> same problem .
> 
>>
>> This should be avoided too, because it is not a "real life test"
>> only run the api against empty "virgin" database tables
>> (e.g. maybe some side-effects only occur when re-run tests against 
>> filled tables).
>>
>> just my 2 cent ;-)
> 
> euro cents ?
> 
>>
>> regards,
>> Armin
> 
> 
> jakob
> 
>>
>>> i run alltests before commiting any change to the repository.
>>>
>>> jakob
>>>
>>>
>>> oliver.matz@ppi.de wrote:
>>>
>>>> Hello again,
>>>>
>>>> I am a little bit annoyed at the moment because
>>>> I spent several hours figuring out how I corrupted
>>>> test cases until I finally discovered that some test failures are 
>>>> not reproducable.
>>>>
>>>> More precisely:
>>>> I checked out current CVS head into an empty directory and then 
>>>> repeated
>>>>
>>>> bin\build.bat junit-no-compile junit-report
>>>>
>>>> several times.  In between I saved the target/test directory. Then I 
>>>> compared the results.  There are various results.  Some
>>>> test fail non-deterministically.  Besides, there
>>>> seem to be dependencies such as:
>>>>
>>>> test Y succeeds iff test X runs before test Y.
>>>>
>>>> Discovering that took quite a while, and I was
>>>> only doing the tests to make sure that some minor
>>>> cleanup has not done any harm.
>>>>
>>>> I wonder if someone else has experienced that?
>>>> How often do you run the tests? In order to benefit from the junit 
>>>> tests, every developer should be able to rely on that all tests that 
>>>> are not marked as 'allowed to fail' do not fail
>>>> unless he or she corrupts them.
>>>>
>>>>
>>>>> -----Original Message-----
>>>>> From: oliver.matz@ppi.de [mailto:oliver.matz@ppi.de]
>>>>> Sent: Monday, November 10, 2003 3:03 PM
>>>>> To: ojb-dev@db.apache.org
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>>> Can I compare these results to the 'official' ones?
>>>>>> The link http://db.apache.org/ojb/junit-report.html
>>>>>> on http://db.apache.org/ojb/maven-reports.html is broken.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> the tests have changed, but this issue is still open.
>>>>>
>>>>> Does someone take care of this?
>>>>>
>>>>> Can we achieve any consensus about how to deal with failing
>>>>> tests?  I have stated my opinion about this in another
>>>>> thread.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> This issue is really important to me.
>>>>
>>>> I will have a look at some tests that fail sometimes.
>>>>
>>>> Olli
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: ojb-dev-unsubscribe@db.apache.org
>>>> For additional commands, e-mail: ojb-dev-help@db.apache.org
>>>>
>>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: ojb-dev-unsubscribe@db.apache.org
>>> For additional commands, e-mail: ojb-dev-help@db.apache.org
>>>
>>>
>>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: ojb-dev-unsubscribe@db.apache.org
>> For additional commands, e-mail: ojb-dev-help@db.apache.org
>>
>>
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: ojb-dev-unsubscribe@db.apache.org
> For additional commands, e-mail: ojb-dev-help@db.apache.org
> 
> 
> 



---------------------------------------------------------------------
To unsubscribe, e-mail: ojb-dev-unsubscribe@db.apache.org
For additional commands, e-mail: ojb-dev-help@db.apache.org


Mime
View raw message