Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@couchdb.apache.org
Received-SPF: pass (athena.apache.org: domain of dch@jsonified.com designates
 209.85.215.49 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <5180BF52.3040605@cego.dk>
References: <5180BF52.3040605@cego.dk>
Date: Wed, 1 May 2013 11:35:27 +0200
Message-ID: 
 <CAL+Y1nvPvZKxMje=uDMGpNAis67xOaJdvr5=pnFAbX0TQt4j4A@mail.gmail.com>
Subject: Re: Replication misses documents
From: Dave Cottlehuber <dch@jsonified.com>
To: user@couchdb.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

On 1 May 2013 09:08, Robert Larsen <rl@cego.dk> wrote:
> Hi all
>
> I have been playing with CouchDB and I am having a weird problem.
> A database I have replicated (and is continuously replicating) contains
> many documents that are not present in the replicated version even thou
> it says that the replication is 100% done.
>
> And even rather old documents are missing (the database I replicate from
> is itself a replication of the Node NPM registry).
>
> Getting the 'underscore' document from the source database:
> $ curl -s -X GET http://192.168.192.70:5984/registry/underscore
> {"_id":"underscore","_rev":"117-eaf1f4e86f1e84f50d5df55fb56998a0","name":=
"underscore","description":"JavaScript's
> functional programming helper
> library.","dist-tags":{"latest":"1.4.4","stable":"1.4.4"},"versions":{"1.=
0.3":{"name":"underscore"........
>
> Viewing the active tasks on the replicated database:
> $ curl -s -X GET http://localhost:5984/_active_tasks
> [{"pid":"<0.9387.5>","checkpointed_source_seq":38146,"continuous":true,"d=
oc_id":null,"doc_write_failures":0,"docs_read":0,"docs_written":0,"missing_=
revisions_found":0,"progress":100,"replication_id":"34a476394d72da3f2b4d85f=
7f8acd50c+continuous","revisions_checked":0,"source":"http://192.168.192.70=
:5984/registry/","source_seq":38146,"started_on":1367329977,"target":"regis=
try","type":"replication","updated_on":1367391827}]
>
> See, 100% replicated.
>
> Getting the 'underscore' document from the replication:
> $ curl -s -X GET http://localhost:5984/registry/underscore
> {"error":"not_found","reason":"missing"}
>
>
> What have I done wrong?

The source_seq is very low, compared to the current npm registry:

curl -s  http://isaacs.iriscouch.com/registry | json_reformat | grep commit=
ted

    "committed_update_seq": 474407

So either you're not pointing at the actual registry, or this
replication is no longer running for some reason.

I'd cancel the replication, and re-create it using the _replicator
endpoint so it gets picked up even if the server restarts. Something
like this (untested):

curl -X PUT http://admin:passwd@localhost:5984/_replicator/npm-mirror
--data-binary '{"source":"https://isaacs.iriscouch.com/registry",
"target":"npm-mirror", "continuous": true}' -H "Content-Type:
application/json"

For the failure, a snippet of the couch.log file will probably explain
why this is failing.

A+
Dave