couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Anderson <>
Subject Re: Using the API to tell if a compact has been done
Date Mon, 24 Jan 2011 21:05:39 GMT
One approach I've been playing with is a patch to couchdb to record
the sequence number at which the last compact was performed, and
adding this value to the db info blob. This lets me get a sense of how
much the database has aged since the last compaction, as well as being
able to tell a compaction has completed successfully by a incremented
compact sequence.

If people are interested I can clean up and post it as a patch to 1.0.x

A little background: I've been working on improving compaction of
couchdb databases in my spare time. Opscode's particular (ab)use case
generates a pretty heavy write update load, with large json blobs
being frequently replaced. This not only causes a very rapid bloat of
the couch files, but often causes load induced crashes of the
compaction process (it seems to be bottlenecking on the couch ref
counter). Knowing if a compaction finished successfully is very
helpful in that circumstance.

On Mon, Jan 24, 2011 at 12:00 PM, Ian Hobson <> wrote:
> On 24/01/2011 17:49, Wayne Conrad wrote:
>> I've subscribed again, from another email address, because the list
>> decided that my first subscribtion was a eeeeevil spammer.  I hope that's
>> OK.  Anyhow...
>> I've written a cron job that periodically compacts all databases on all
>> servers, but it needs an automated test, and an automated test needs a way
>> to know that compaction happend (or, at least, was triggered).  Other than
>> seeing the file size change, is there anything I can do from the API that
>> will show that my cron job caused compaction to happen?
> Hi Wayne,
> If you can create the database you are to compact, then you can use the
> rev=??? parameter.
> Set up the database, and note the rev of a document  (= oldrev)
> Update the document (creating _rev=newrev). oldrev is now on disk, but not
> normally available.
> Read the old version using the rev=oldrev parameter.
> Trigger the compaction and wait for it to finish (A 1 record database won't
> take long to compact :) )
> Read the old record again. This time it won't be found if the compaction has
> run.
> See
> Regards
> Ian

View raw message