couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jan Lehnardt <...@apache.org>
Subject Re: Test suite blocking release
Date Fri, 19 Mar 2010 16:49:12 GMT

On 19 Mar 2010, at 11:44, Noah Slater wrote:

> Yep, I checked both browsers out of habit more than anything.
> 
> What about the other thread discussing an actual issue brought up by some of these failures?

It was an issue in the test case, not core code.


> Is it not worrying that they failed first then worked, or is this something ignorable?

The latter.


> And if so, what can we do to fix the whole situation post-release?

(answering anyway :) Move all JS tests to etap tests to factor out browser 
issues, caching, timing, etc and don't introduce any errors.


Cheers
Jan
--


> 
> On 19 Mar 2010, at 15:53, Jan Lehnardt wrote:
> 
>> Hi Noah,
>> 
>> The test suite doesn't support Safari. While I agree that the first-failing-then-working
tests in Firefox are concerning, I say it doesn't block a release.
>> 
>> Cheers
>> Jan
>> --
>> 
>> 
>> On 19 Mar 2010, at 06:59, Noah Slater wrote:
>> 
>>> Hey,
>>> 
>>> I was going to call a vote on the release today, but I am being blocked by test
suite errors.
>>> 
>>> Running in Safari:
>>> 
>>> 	changes	
>>> 
>>> 	• Assertion 'should return matching rows' failed: expected '3', got '1'
>>> 
>>> 	rev_stemming
>>> 
>>> 	• Assertion 'should return a truncated revision list' failed: expected '5',
got '6'
>>> 
>>> The first one fails consistently, with debugging and without. No additional details
provided by the test.
>>> 
>>> The second one is quite mysterious, however, and needs to be fixed.
>>> 
>>> I ran it again with the debugger, and IT PASSED.
>>> 
>>> I ran it again as normal, and got:
>>> 
>>> 	• Assertion failed: db.open("bar", {revs:true})._revisions.ids.length == newLimit
+ 1
>>> 	• Assertion failed: docB2._conflicts[0] == docB1._rev) // We having already
updated bar before setting the limit, so it's still got // a long rev history. compact to
stem the revs. T(db.open("bar", {revs:true})._revisions.ids.length == newLimit + 1
>>> 
>>> I ran it again with the debugger:
>>> 
>>> 	• Assertion failed: false
>>> 	• Assertion failed: false
>>> 
>>> I ran the whole thing from scratch, and I got the first error, then the second
error message for the second test.
>>> 
>>> Running on Firefox:
>>> 
>>> 	changes
>>> 
>>> 	• Assertion 'JSON.parse(lines[2]).id == "rusty", lines[2]' failed: {"last_seq":9}
>>> 	• Exception raised: {"message":"JSON.parse","fileName":"http://127.0.0.1:5984/_utils/script/couch_test_runner.js?0.11.0","lineNumber":167,"stack":"(false)@http://127.0.0.1:5984/_utils/script/couch_test_runner.js?0.11.0:167\u000arun(11)@http://127.0.0.1:5984/_utils/script/couch_test_runner.js?0.11.0:83\u000a"}
>>> compactsuccess304ms
>>> 
>>> 	list_views
>>> 
>>> 	• Exception raised: {}
>>> 
>>> Both tests pass when run a second time.
>>> 
>>> A few other people on IRC were able to reproduce some of the errors I have described
here.
>>> 
>>> I now consider these tests suspicious too, and have no idea how serious the problems
are.
>>> 
>>> Any help or guidance would be fantastic.
>>> 
>>> We're so close to the release!
>>> 
>> 
> 


Mime
View raw message