incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steven Hazel <...@saucelabs.com>
Subject Re: An asynchronous write bug in CouchDB 0.9?
Date Mon, 27 Apr 2009 17:49:45 GMT
Okay, I think I've confirmed this theory.  Thanks to your guidance I managed
to create a python script that can reproduce the problem on my macbook:
http://saucelabs.com/couch-bug.py

(To run it, you need the latest trunk version of couchdb-python.)

Looking at our logs, we started seeing this problem just after we crossed
the threshold to >100 dbs.  And very frequent reads to the problematic
database fixed the issue, because they made it always more recently active
than most of the others.

- Steve

On Mon, Apr 27, 2009 at 9:40 AM, Steven Hazel <sah@saucelabs.com> wrote:

> I think we do have a lot of active databases -- we have a view that is run
> frequently against each of our 137 databases.
>
> - Steve
>
>
> On Mon, Apr 27, 2009 at 6:20 AM, Damien Katz <damien@apache.org> wrote:
>
>> It looks like the db server instance is being restarted before it commits
>> the header, but it's odd you aren't seeing any errors on the console. If
>> it's a crash it should dump an error to the console.
>>
>> One thing that occurs to me, do you have a lot of active databases (> 100)
>> ? I think it's possible when you hit the max dbs open, CouchDB will shutdown
>> an idle instance with pending deferred commit, which is a bug.
>>
>> -Damien
>>
>>
>>
>> On Apr 27, 2009, at 12:52 AM, Steven Hazel wrote:
>>
>>  Hi,
>>>
>>> Last week we discovered that records in one of our CouchDB databases were
>>> occasionally being reverted to the previous revision. We believe this is
>>> due
>>> to a bug in CouchDB 0.9 related to asynchronous writes.  Turning off
>>> asynchronous writes with the X-Couch-Full-Commit header fixes the problem
>>> for us.  I've written up the problem in more detail here:
>>> http://saucelabs.com/couch-bug.html
>>>
>>> We haven't yet reproduced this problem outside our production
>>> environment,
>>> but can trigger it there on-demand.  Although we have a work-around that
>>> solves the immediate problem for us, we're happy to help out in tracking
>>> down the underlying issue.  Any advice on how to proceed would be
>>> appreciated.
>>>
>>> --
>>> Steven Hazel
>>> Sauce Labs
>>> Cofounder and Director, Development
>>> http://twitter.com/sahazel
>>>
>>
>>
>
>
> --
> Steven Hazel
> Sauce Labs
> Cofounder and Director, Development
> http://twitter.com/sahazel
>



-- 
Steven Hazel
Sauce Labs
Cofounder and Director, Development
http://twitter.com/sahazel

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message