couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mathias Leppich <mlepp...@googlemail.com>
Subject Re: Using the API to tell if a compact has been done
Date Tue, 25 Jan 2011 07:23:29 GMT
Awesome! I'm interested in your patch. The idea is pretty straight fwd.

I think knowing that compaction has finished, and how many changes are
made since, should make it's way into the couchdb source tree.

-mathias

Am 24.01.2011 um 22:06 schrieb Mark Anderson <mark@opscode.com>:

> One approach I've been playing with is a patch to couchdb to record
> the sequence number at which the last compact was performed, and
> adding this value to the db info blob. This lets me get a sense of how
> much the database has aged since the last compaction, as well as being
> able to tell a compaction has completed successfully by a incremented
> compact sequence.
>
> If people are interested I can clean up and post it as a patch to 1.0.x
>
> A little background: I've been working on improving compaction of
> couchdb databases in my spare time. Opscode's particular (ab)use case
> generates a pretty heavy write update load, with large json blobs
> being frequently replaced. This not only causes a very rapid bloat of
> the couch files, but often causes load induced crashes of the
> compaction process (it seems to be bottlenecking on the couch ref
> counter). Knowing if a compaction finished successfully is very
> helpful in that circumstance.
>
>
>
>
>
>
>
>
> On Mon, Jan 24, 2011 at 12:00 PM, Ian Hobson <ian@ianhobson.co.uk> wrote:
>> On 24/01/2011 17:49, Wayne Conrad wrote:
>>>
>>> I've subscribed again, from another email address, because the list
>>> decided that my first subscribtion was a eeeeevil spammer.  I hope that's
>>> OK.  Anyhow...
>>>
>>> I've written a cron job that periodically compacts all databases on all
>>> servers, but it needs an automated test, and an automated test needs a way
>>> to know that compaction happend (or, at least, was triggered).  Other than
>>> seeing the file size change, is there anything I can do from the API that
>>> will show that my cron job caused compaction to happen?
>>>
>> Hi Wayne,
>>
>> If you can create the database you are to compact, then you can use the
>> rev=??? parameter.
>>
>> Set up the database, and note the rev of a document  (= oldrev)
>> Update the document (creating _rev=newrev). oldrev is now on disk, but not
>> normally available.
>>
>> Read the old version using the rev=oldrev parameter.
>>
>> Trigger the compaction and wait for it to finish (A 1 record database won't
>> take long to compact :) )
>>
>> Read the old record again. This time it won't be found if the compaction has
>> run.
>>
>> See
>> http://wiki.apache.org/couchdb/HTTP_Document_API#Accessing_Previous_Revisions
>>
>> Regards
>>
>> Ian
>>
>>
>>

Mime
View raw message