couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Anderson <>
Subject Re: Tail Append Headers
Date Fri, 22 May 2009 20:44:30 GMT
On Fri, May 22, 2009 at 1:26 PM, Damien Katz <> wrote:
> Chris Anderson has done some work here with the batch feature for similar
> efficiency reasons, except that his feature is to buffer the docs into
> memory and return success to the client immediately, it's for things where
> losing some messages is acceptable (like logging), so long as the
> transactions are fast and efficient as possible. You instead want the client
> to wait and get the success or fail. it seems the batch work is where to
> build this into.

Exactly. It'd be a "trivial" change to the batch=ok feature to add
wait=true so that the client hangs open on the connection and receives
a 201 Created when the batch is committed. It'd actually be sort of a
challenging patch, but I can perhaps add it before 1.0.

I've added it to Jira here:


> -Damien
> On May 22, 2009, at 4:08 AM, Brian Candler wrote:
>> On Wed, May 20, 2009 at 11:48:10AM -0400, Damien Katz wrote:
>>>> As for timing of fsync: ideally what I would like is for each write
>>>> operation to return some sort of checkpoint tag (which could just be
>>>> the
>>>> current file size). Then have a new HTTP operation "wait for sync
>>>> <tag>"
>>>> which blocks until the file has been fsync'd at or after that
>>>> position.
>>> Just use the x-couch-full-commit=true http header.
>> That's not what I want (unless I've misunderstood what this header does).
>> Suppose I have 100 concurrent client connections, each client is
>> committing
>> one document at roughly the same time. I don't one 100 fsync's; I want
>> each
>> client to be able to write a doc, and then be notified when the next
>> convenient fsync takes place, so they know their doc is on persistent
>> storage.
>> Setting the default commit interval down to (say) 200ms would then give
>> clients a reasonably low latency for getting their docs committed to disk,
>> without killing performance by fsyncing every write.
>> Otherwise, the only solution I can see is a batching proxy: it takes PUT
>> and
>> POST requests, builds them into a single _bulk_docs, and submits that
>> periodically with x-couch-full-commit:true before responding. But that
>> gets
>> very awkward when dealing with multiple databases, and/or multiple users
>> with different credentials.
>> Regards,
>> Brian.

Chris Anderson

View raw message