couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Anderson <>
Subject Re: Tail Append Headers
Date Fri, 22 May 2009 20:46:54 GMT
On Fri, May 22, 2009 at 1:44 PM, Chris Anderson <> wrote:
> On Fri, May 22, 2009 at 1:26 PM, Damien Katz <> wrote:
>> Chris Anderson has done some work here with the batch feature for similar
>> efficiency reasons, except that his feature is to buffer the docs into
>> memory and return success to the client immediately, it's for things where
>> losing some messages is acceptable (like logging), so long as the
>> transactions are fast and efficient as possible. You instead want the client
>> to wait and get the success or fail. it seems the batch work is where to
>> build this into.
> Exactly. It'd be a "trivial" change to the batch=ok feature to add
> wait=true so that the client hangs open on the connection and receives
> a 201 Created when the batch is committed. It'd actually be sort of a
> challenging patch, but I can perhaps add it before 1.0.
> I've added it to Jira here:

Forgot to mention that you can almost get this now by using batch=ok
and then when the client wants to be sure that the batch is committed,
send a POST to /db/_ensure_full_commit

You'll have a little more http overhead this way but the semantics are
mostly the same.

> Chris
>> -Damien
>> On May 22, 2009, at 4:08 AM, Brian Candler wrote:
>>> On Wed, May 20, 2009 at 11:48:10AM -0400, Damien Katz wrote:
>>>>> As for timing of fsync: ideally what I would like is for each write
>>>>> operation to return some sort of checkpoint tag (which could just be
>>>>> the
>>>>> current file size). Then have a new HTTP operation "wait for sync
>>>>> <tag>"
>>>>> which blocks until the file has been fsync'd at or after that
>>>>> position.
>>>> Just use the x-couch-full-commit=true http header.
>>> That's not what I want (unless I've misunderstood what this header does).
>>> Suppose I have 100 concurrent client connections, each client is
>>> committing
>>> one document at roughly the same time. I don't one 100 fsync's; I want
>>> each
>>> client to be able to write a doc, and then be notified when the next
>>> convenient fsync takes place, so they know their doc is on persistent
>>> storage.
>>> Setting the default commit interval down to (say) 200ms would then give
>>> clients a reasonably low latency for getting their docs committed to disk,
>>> without killing performance by fsyncing every write.
>>> Otherwise, the only solution I can see is a batching proxy: it takes PUT
>>> and
>>> POST requests, builds them into a single _bulk_docs, and submits that
>>> periodically with x-couch-full-commit:true before responding. But that
>>> gets
>>> very awkward when dealing with multiple databases, and/or multiple users
>>> with different credentials.
>>> Regards,
>>> Brian.
> --
> Chris Anderson

Chris Anderson

View raw message