couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Anderson <m...@opscode.com>
Subject Re: Using the API to tell if a compact has been done
Date Mon, 24 Jan 2011 21:05:39 GMT
One approach I've been playing with is a patch to couchdb to record
the sequence number at which the last compact was performed, and
adding this value to the db info blob. This lets me get a sense of how
much the database has aged since the last compaction, as well as being
able to tell a compaction has completed successfully by a incremented
compact sequence.

If people are interested I can clean up and post it as a patch to 1.0.x

A little background: I've been working on improving compaction of
couchdb databases in my spare time. Opscode's particular (ab)use case
generates a pretty heavy write update load, with large json blobs
being frequently replaced. This not only causes a very rapid bloat of
the couch files, but often causes load induced crashes of the
compaction process (it seems to be bottlenecking on the couch ref
counter). Knowing if a compaction finished successfully is very
helpful in that circumstance.








On Mon, Jan 24, 2011 at 12:00 PM, Ian Hobson <ian@ianhobson.co.uk> wrote:
> On 24/01/2011 17:49, Wayne Conrad wrote:
>>
>> I've subscribed again, from another email address, because the list
>> decided that my first subscribtion was a eeeeevil spammer.  I hope that's
>> OK.  Anyhow...
>>
>> I've written a cron job that periodically compacts all databases on all
>> servers, but it needs an automated test, and an automated test needs a way
>> to know that compaction happend (or, at least, was triggered).  Other than
>> seeing the file size change, is there anything I can do from the API that
>> will show that my cron job caused compaction to happen?
>>
> Hi Wayne,
>
> If you can create the database you are to compact, then you can use the
> rev=??? parameter.
>
> Set up the database, and note the rev of a document  (= oldrev)
> Update the document (creating _rev=newrev). oldrev is now on disk, but not
> normally available.
>
> Read the old version using the rev=oldrev parameter.
>
> Trigger the compaction and wait for it to finish (A 1 record database won't
> take long to compact :) )
>
> Read the old record again. This time it won't be found if the compaction has
> run.
>
> See
> http://wiki.apache.org/couchdb/HTTP_Document_API#Accessing_Previous_Revisions
>
> Regards
>
> Ian
>
>
>

Mime
View raw message