couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jens Alfke <j...@couchbase.com>
Subject Re: High latency (40ms) and low request rate (10 rps) under windows
Date Tue, 25 Oct 2011 21:34:43 GMT

On Oct 25, 2011, at 2:19 PM, Konstantin Cherkasov wrote:

"OMG! My database do 10 requests per second (for ex. Postgres do 1000 rps on the same hardware
with fsync on)
"Forget this, just use BATCH MODE"

It’s probably not the same fsync, so you’re comparing apples and oranges. There is no
way a full fsync (pushing all writes onto the disk platters) can be done 1000x a second; 10
is closer to what I’ve seen.

This is a tricky topic. If you just call fsync, you get a “lazy” sync where the kernel
pushes writes to the disk controller but doesn’t wait for them to be written; so they just
end up in the controller’s cache until sometime in the future, not actually in persistent
storage yet. To get a “full” fsync that includes a flush command to the controller, you
have to do some extra work (the API depends on the OS.)

CouchDB does a full sync, because this is the only way to be proof against disasters like
power failures, but there is an option to turn it off. (I think there is a writeup on the
CouchDB wiki about this.)

The same thing happens with sqlite, by the way, if its full-fsync option is on (it is by default
on OS X, not sure of other platforms.) I have seen with my own two eyes the kind of fatal
corruption of sqlite databases that you can get if full-fsync isn’t enabled and a power
failure or kernel panic occurs in the middle of a commit; it’s not pretty.

—Jens

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message