Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@couchdb.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
MIME-Version: 1.0
In-Reply-To: <4D3DDA42.1040807@ianhobson.co.uk>
References: <4D3DBBBE.30902@databill.com>
	<4D3DDA42.1040807@ianhobson.co.uk>
Date: Mon, 24 Jan 2011 13:05:39 -0800
Message-ID: <AANLkTinid7ya9Wva+yzNzRXaKVF-3m4QcnPG9mhk9AD1@mail.gmail.com>
Subject: Re: Using the API to tell if a compact has been done
From: Mark Anderson <mark@opscode.com>
To: user@couchdb.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

One approach I've been playing with is a patch to couchdb to record
the sequence number at which the last compact was performed, and
adding this value to the db info blob. This lets me get a sense of how
much the database has aged since the last compaction, as well as being
able to tell a compaction has completed successfully by a incremented
compact sequence.

If people are interested I can clean up and post it as a patch to 1.0.x

A little background: I've been working on improving compaction of
couchdb databases in my spare time. Opscode's particular (ab)use case
generates a pretty heavy write update load, with large json blobs
being frequently replaced. This not only causes a very rapid bloat of
the couch files, but often causes load induced crashes of the
compaction process (it seems to be bottlenecking on the couch ref
counter). Knowing if a compaction finished successfully is very
helpful in that circumstance.


On Mon, Jan 24, 2011 at 12:00 PM, Ian Hobson <ian@ianhobson.co.uk> wrote:
> On 24/01/2011 17:49, Wayne Conrad wrote:
>>
>> I've subscribed again, from another email address, because the list
>> decided that my first subscribtion was a eeeeevil spammer. =A0I hope tha=
t's
>> OK. =A0Anyhow...
>>
>> I've written a cron job that periodically compacts all databases on all
>> servers, but it needs an automated test, and an automated test needs a w=
ay
>> to know that compaction happend (or, at least, was triggered). =A0Other =
than
>> seeing the file size change, is there anything I can do from the API tha=
t
>> will show that my cron job caused compaction to happen?
>>
> Hi Wayne,
>
> If you can create the database you are to compact, then you can use the
> rev=3D??? parameter.
>
> Set up the database, and note the rev of a document =A0(=3D oldrev)
> Update the document (creating _rev=3Dnewrev). oldrev is now on disk, but =
not
> normally available.
>
> Read the old version using the rev=3Doldrev parameter.
>
> Trigger the compaction and wait for it to finish (A 1 record database won=
't
> take long to compact :) )
>
> Read the old record again. This time it won't be found if the compaction =
has
> run.
>
> See
> http://wiki.apache.org/couchdb/HTTP_Document_API#Accessing_Previous_Revis=
ions
>
> Regards
>
> Ian
>
>
>