couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Damien Katz <dam...@apache.org>
Subject Faster updates, optional ACID
Date Mon, 05 Jan 2009 17:54:58 GMT
It was brought to my attention that commits on OS X were very slow  
with the latest releases of Erlang. After I upgraded to the most  
recent version, I found them to be indeed slow, slowing the tests down  
to point it was painful to run them. It appears, any disk sync from  
Erlang now takes somewhere between 50 and 100ms, up from the previous  
times of ~5ms. This is almost certainly due to the F_FULLFSYNC flag  
the erlang file handling now uses on darwin based systems, but it's  
surprising how bad performance on OS X. A little investigation had  
shown other database engines have similar issues on OS X.

To address this problem, I implemented delayed commit functionality.  
We had always intended to implement delayed commit for performance  
reasons, but hadn't had the need until now. This makes updates much  
faster in the general case, but with the caveat they aren't flushed  
completely to disk right way. If you can't tolerate the possible loss  
of recent updates, you can use the "full commit" option for ACID  
commits.

For full acid commit, add a header field to the doc PUT or _bulk_docs  
POST like this:
  X-Couch-Full-Commit:true

Then couchdb will completely commit the change before returning.

Also, if you have several delayed updates and you want to make sure  
they all made it to disk, you can invoke POST /db/_ensure_full_commit  
and all outstanding commits are flushed to disk.

The view engine has been already modified to deal with delayed commits  
too, it ensures it never fully commits it's own indexes to disk if the  
documents indexed aren't already committed to disk.

The last remaining work item is db server crash detection, so that  
clients can detect when a server has crashed and potentially lost  
updates. This is pretty simple, each db server just needs a unique ID  
generated at it's startup. Client retrieve this value at the beginning  
of the writes and then checks that the value is the same once down a  
flushed to disk. If not, we know we maybe have lost some updates and  
we redo the replication from the last known good commit.

Right now the default is to delay the commits, because I think that  
will be the most common use case but I'm really not sure. I definitely  
want the commits delayed for the test suite, to keep things running  
fast.

-Damien



Mime
View raw message