couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Yves Moulin <...@baaz.fr>
Subject replication slow down
Date Tue, 20 Aug 2013 12:49:50 GMT
Hi CouchDB users,

I'm trying to replicate a big database: 35 millions documents. Average doc size is 500 bytes.

When replication starts, from server A to server B (from server B in pull mode), we got replicate
at 1100 documents per second. Then, after some thousands of documents replicated, replication
slowly decelerates until 100 doc/sec.

If I restart the couchdb daemon on server B, replication restarts at 1100 doc/sec. And then
slow down again. At 100 doc/sec I need multiple weeks to replicate the full database.. :-(

Obviously, the two servers are far from their performances limits: one half CPU used (on 16),
10GB of RAM used (on 64GB), and very small I/O (on 24 disks + 2 ssd for cache).

You can see here: http://jym.eileo.net/couchdb_req.png , A small graph showing the http requests
on server A (almost all GET requests come from server B). The restart of server B is clearly
visible at 10:30.

I already try to change worker_batch_size, worker_processes and http_connections but this
doesn't change anything.

Do you know if this is a know bug or how can I improve the situation ?


Thank you very much.

best,
jym
Mime
View raw message