incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Max Ogden <...@maxogden.com>
Subject Re: CouchDB Crash report db_not_found when attempting to replicate databases
Date Tue, 13 Sep 2011 18:44:54 GMT
Hi Chris,

>From what I understand the current state of the replicator (as of 1.1) is
that for certain types of collections of documents it can be somewhat
fragile. In the case of the node.js package repository, http://npmjs.org,
there are many relatively large (~100MB) documents that would sometimes
throw errors or timeout during replication and crash the replicator, at
which point the replicator would restart and attempt to pick up where it
left off. I am not an expert in the internals of the replicator but
apparently the cumulative time required for the replicator to repeatedly
crash and then subsequently relocate itself in _changes feed in the case of
replicating the node package manager was making the built in couch
replicator unusable for the task.

Two solutions exist that I know of. There is a new replicator in trunk (not
to be confused with the _replicator db from 1.1 -- it is still using the old
replicator algorithms) and there is also a more reliable replicator written
in node.js https://github.com/mikeal/replicate that was was written
specifically to replicate the node package repository between hosting
providers.

Additionally it may be useful if you could describe the 'fingerprint' of
your documents a bit. How many documents are in the failing databases? are
the documents large or small? do they have many attachments? how large is
your _changes feed?

Cheers,

Max

On Tue, Sep 13, 2011 at 11:22 AM, Chris Stockton
<chrisstocktonaz@gmail.com>wrote:

> Hello,
>
> We now have about 150 dbs that are refusing to replicate with random
> crashes, which provide really zero debug information. The error is db
> not found, but I know its available. Does anyone know how can I
> trouble shoot this? Do we just have to many databases replicating for
> couchdb to handle? 4000 is a small number for the massive hardware
> these are running on.
>
> -Chris
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message