couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Newson (JIRA)" <>
Subject [jira] Commented: (COUCHDB-487) Pause write requests to allow compaction to complete
Date Thu, 27 Aug 2009 13:23:59 GMT


Robert Newson commented on COUCHDB-487:

That's a fair concern. My preferred fix is to use multiple files that can be compacted/obsoleted
separately, which makes it non-blocking even under heavy nd continuous writing. 

I was aware of the multi-pass nature of compaction when I wrote the patch, which is why it
doesn't kick in at all unless it tries and fails to switch over to the compact file. I can
modify the patch to only make write_available=false if the seq delta is below a configurable
threshold, which is a nice change anyway, and the default can be 0, which is today's behavior.

> Pause write requests to allow compaction to complete
> ----------------------------------------------------
>                 Key: COUCHDB-487
>                 URL:
>             Project: CouchDB
>          Issue Type: Improvement
>          Components: Database Core
>            Reporter: Robert Newson
>         Attachments: block_writes_after_end_of_compaction.patch
> Continuous writes will currently indefinitely prevent the compaction process from flipping
over to the .compact file.
> Here's a small patch that adds a new db flag called 'write_available' which becomes false
the first time compaction completes but fails to flip over because of concurrent writes. Subsequent
calls to update_docs then sleep for 1/2 a second and throw retry.
> I ran 'ab -p json -n 1000000 http://localhost:5984/db1' in one window. Without this patch,
I get a long sequence of;
> [info] [<0.64.0>] Compaction file still behind main file (update seq=1848. compact
update seq=1845). Retrying.
> and compaction never completes.
> With the patch, I get this;
> [info] [<0.369.0>] - - 'POST' /db1 201
> [info] [<0.65.0>] Compaction file still behind main file (update seq=2140. compact
update seq=2092). Retrying.
> [info] [<0.65.0>] Blocking writes to complete compaction.
> [info] [<0.65.0>] Compaction for db "db1" completed.
> [info] [<0.370.0>] - - 'POST' /db1 201

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message