incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adam Kocoloski <kocol...@apache.org>
Subject Re: replication slow down
Date Tue, 20 Aug 2013 13:16:22 GMT
Did you specify "continuous": true in the replication spec?  If you did, do you see the same
slowdown if you omit that flag?

This sounds rather similar to a bug we encountered in early builds of BigCouch, but I don't
remember the history of the codebase well enough to say whether the bug might be latent in
Apache CouchDB.

Adam

On Aug 20, 2013, at 8:49 AM, Jean-Yves Moulin <jym@baaz.fr> wrote:

> Hi CouchDB users,
> 
> I'm trying to replicate a big database: 35 millions documents. Average doc size is 500
bytes.
> 
> When replication starts, from server A to server B (from server B in pull mode), we got
replicate at 1100 documents per second. Then, after some thousands of documents replicated,
replication slowly decelerates until 100 doc/sec.
> 
> If I restart the couchdb daemon on server B, replication restarts at 1100 doc/sec. And
then slow down again. At 100 doc/sec I need multiple weeks to replicate the full database..
:-(
> 
> Obviously, the two servers are far from their performances limits: one half CPU used
(on 16), 10GB of RAM used (on 64GB), and very small I/O (on 24 disks + 2 ssd for cache).
> 
> You can see here: http://jym.eileo.net/couchdb_req.png , A small graph showing the http
requests on server A (almost all GET requests come from server B). The restart of server B
is clearly visible at 10:30.
> 
> I already try to change worker_batch_size, worker_processes and http_connections but
this doesn't change anything.
> 
> Do you know if this is a know bug or how can I improve the situation ?
> 
> 
> Thank you very much.
> 
> best,
> jym


Mime
View raw message