Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@couchdb.apache.org
Received-SPF: pass (nike.apache.org: domain of randall.leeds@gmail.com
 designates 209.85.214.180 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type:content-transfer-encoding;
        b=dfgueFNKXn5ugTuNpKihGpkjPMySy53pzKKkGTW/kZyRtcJ1fS6Hq8LX77G655sCUl
         vrSt4QAWzmPnJAyVGksroe/mVhQQxybPHmtDS/pBzt/om9oB+VNCh5dv6NlSZHK2H6qN
         CIgBPQ+1K98W37gyDxpViLbWMuC35R1RORkAA=
MIME-Version: 1.0
In-Reply-To: <4D8CF928.9000506@databill.com>
References: <4D8CF928.9000506@databill.com>
Date: Fri, 25 Mar 2011 13:25:07 -0700
Message-ID: <AANLkTintWijqK0e7f=5FGbdb1mycfQtHGzGCEAiRH96a@mail.gmail.com>
Subject: Re: What causes compaction to retry?
From: Randall Leeds <randall.leeds@gmail.com>
To: user@couchdb.apache.org
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

On Fri, Mar 25, 2011 at 13:20, Wayne Conrad <wayne@databill.com> wrote:
> I've got a largish database that is being both compacted and synchronized
> concurrently.
>
> $ curl --silent topaz:5984/_active_tasks | prettify_json.rb
> [
> =C2=A0{
> =C2=A0 =C2=A0"pid": "<0.313.0>",
> =C2=A0 =C2=A0"type": "Replication",
> =C2=A0 =C2=A0"task": "`874dbdaf26454065f33007eb42c8320a+create_target`:
> `http://sodium:5984/incoming/` -> `incoming`",
> =C2=A0 =C2=A0"status": "Processed source seq 477428"
> =C2=A0},
> =C2=A0{
> =C2=A0 =C2=A0"pid": "<0.24865.58>",
> =C2=A0 =C2=A0"type": "Database Compaction",
> =C2=A0 =C2=A0"task": "incoming retry",
> =C2=A0 =C2=A0"status": "Copied 106794 of 159518 changes (66%)"
> =C2=A0}
> ]
>
> At first the task was just "incoming", but when it got to 100%, it starte=
d
> over again with the task now being "incoming retry." =C2=A0It's gotten to=
 the end
> a few times now, each time starting over as a "retry" task. =C2=A0What ca=
uses
> that?
>

When compacting, Couch traverses a snapshot of the database and copies
the current revisions out to a new, compacted database file.
When it reaches the end, it needs to incorporate any new revisions
that happened during the first pass.
Each successive retry should be much faster than the last.
In theory, if the database is busy enough, compaction might never
finish. In practice, I don't think anyone has ever seen this occur.

R