couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Damien Katz <dam...@apache.org>
Subject Re: Tail Append Headers
Date Fri, 22 May 2009 20:26:18 GMT
Chris Anderson has done some work here with the batch feature for  
similar efficiency reasons, except that his feature is to buffer the  
docs into memory and return success to the client immediately, it's  
for things where losing some messages is acceptable (like logging), so  
long as the transactions are fast and efficient as possible. You  
instead want the client to wait and get the success or fail. it seems  
the batch work is where to build this into.

-Damien


On May 22, 2009, at 4:08 AM, Brian Candler wrote:

> On Wed, May 20, 2009 at 11:48:10AM -0400, Damien Katz wrote:
>>> As for timing of fsync: ideally what I would like is for each write
>>> operation to return some sort of checkpoint tag (which could just be
>>> the
>>> current file size). Then have a new HTTP operation "wait for sync
>>> <tag>"
>>> which blocks until the file has been fsync'd at or after that
>>> position.
>>
>> Just use the x-couch-full-commit=true http header.
>
> That's not what I want (unless I've misunderstood what this header  
> does).
>
> Suppose I have 100 concurrent client connections, each client is  
> committing
> one document at roughly the same time. I don't one 100 fsync's; I  
> want each
> client to be able to write a doc, and then be notified when the next
> convenient fsync takes place, so they know their doc is on persistent
> storage.
>
> Setting the default commit interval down to (say) 200ms would then  
> give
> clients a reasonably low latency for getting their docs committed to  
> disk,
> without killing performance by fsyncing every write.
>
> Otherwise, the only solution I can see is a batching proxy: it takes  
> PUT and
> POST requests, builds them into a single _bulk_docs, and submits that
> periodically with x-couch-full-commit:true before responding. But  
> that gets
> very awkward when dealing with multiple databases, and/or multiple  
> users
> with different credentials.
>
> Regards,
>
> Brian.


Mime
View raw message