incubator-couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Adam Kocoloski (JIRA)" <j...@apache.org>
Subject [jira] Created: (COUCHDB-160) replication performance improvements
Date Tue, 11 Nov 2008 18:53:44 GMT
replication performance improvements
------------------------------------

                 Key: COUCHDB-160
                 URL: https://issues.apache.org/jira/browse/COUCHDB-160
             Project: CouchDB
          Issue Type: Improvement
          Components: Database Core
    Affects Versions: 0.9
            Reporter: Adam Kocoloski
            Priority: Minor


I wrote some code to speed up CouchDB's replication process by parallelizing document requests
and using _bulk_docs to write changes to the target.  I tested the speedup as follows:

* 1000 document DB, 1022 update_seq, ~450 KB after compaction
* local and remote machines have ~45 ms latency
* timed requests using timer:tc(couch_rep, replicate, [<<"source">>, <<"target">>]
* all replications are "from scratch"

trunk:
local-local     115
local-remote    145
remote-remote   173
remote-local    146
db size after replication: 1.8 MB

patch:
local-local     1.83
local-remote    38
remote-remote   64
remote-local    35
db size after replication: 453 KB

I'll attach the patch as an update to this issue.  It might be worth exposing the "batch size"
(currently 100 docs) as a configurable parameter.  Comments welcome.  Best, 

Adam



-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message