couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brian Candler <B.Cand...@pobox.com>
Subject Re: Tail Append Headers
Date Fri, 22 May 2009 08:08:39 GMT
On Wed, May 20, 2009 at 11:48:10AM -0400, Damien Katz wrote:
>> As for timing of fsync: ideally what I would like is for each write
>> operation to return some sort of checkpoint tag (which could just be  
>> the
>> current file size). Then have a new HTTP operation "wait for sync  
>> <tag>"
>> which blocks until the file has been fsync'd at or after that  
>> position.
>
> Just use the x-couch-full-commit=true http header.

That's not what I want (unless I've misunderstood what this header does).

Suppose I have 100 concurrent client connections, each client is committing
one document at roughly the same time. I don't one 100 fsync's; I want each
client to be able to write a doc, and then be notified when the next
convenient fsync takes place, so they know their doc is on persistent
storage.

Setting the default commit interval down to (say) 200ms would then give
clients a reasonably low latency for getting their docs committed to disk,
without killing performance by fsyncing every write.

Otherwise, the only solution I can see is a batching proxy: it takes PUT and
POST requests, builds them into a single _bulk_docs, and submits that
periodically with x-couch-full-commit:true before responding. But that gets
very awkward when dealing with multiple databases, and/or multiple users
with different credentials.

Regards,

Brian.

Mime
View raw message