couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Newson (Commented) (JIRA)" <>
Subject [jira] [Commented] (COUCHDB-1342) Asynchronous file writes
Date Thu, 17 Nov 2011 12:29:52 GMT


Robert Newson commented on COUCHDB-1342:

@Damien, thanks for the clarification. It's possible to read what you originally wrote as
an intention to dismiss concerns over introducing further complexity as long as it improved
performance. Since Paul has now explicitly described several technical problems with the patch
I'm sure we can all move on to addressing them. I'll only add that I had read the entire page
on voting and didn't feel that the section you highlighted applied in this case, which is
why I didn't mention it myself.

I would like to know why couch_file now needs two file descriptors to provide asynchronous
writes, though. If it's integral, I'd appreciate knowing why, and whether there are alternatives.
If not, a separate ticket seems appropriate. The only thing I can think of is the inability
to usefully pass a handle to a file opened with [raw] between processes. In any case, doubling
the consumption of precious server resources should be called out explicitly, rather than
incidentally, in my opinion.

Am I also right in thinking that the improved write performance entails increased memory usage
(and therefore increased GC too)? What's the impact of that on very busy servers?

> Asynchronous file writes
> ------------------------
>                 Key: COUCHDB-1342
>                 URL:
>             Project: CouchDB
>          Issue Type: Improvement
>          Components: Database Core
>            Reporter: Jan Lehnardt
>             Fix For: 1.3
>         Attachments: COUCHDB-1342.patch
> This change updates the file module so that it can do
> asynchronous writes. Basically it replies immediately
> to process asking to write something to the file, with
> the position where the chunks will be written to the
> file, while a dedicated child process keeps collecting
> chunks and write them to the file (and batching them
> when possible). After issuing a series of write request
> to the file module, the caller can call its 'flush'
> function which will block the caller until all the
> chunks it requested to write are effectively written
> to the file.
> This maximizes the IO subsystem, as for example, while
> the updater is traversing and modifying the btrees and
> doing CPU bound tasks, the writes are happening in
> parallel.
> Originally described at
> Github Commit:

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


View raw message