couchdb-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
Subject [3/6] couch-replicator commit: updated refs/heads/integration to ea59d11
Date Mon, 12 May 2014 15:16:52 GMT
remove Cloudant refs


Branch: refs/heads/integration
Commit: b38025c2db4322f5715c0ad54d1754c847cf9e9e
Parents: a63dc61
Author: Robert Newson <>
Authored: Mon May 12 13:09:33 2014 +0100
Committer: Robert Newson <>
Committed: Mon May 12 15:58:29 2014 +0100

----------------------------------------------------------------------                    | 200 --------------------------------------
 src/couch_replicator_sup.erl |   1 -
 2 files changed, 201 deletions(-)
diff --git a/ b/
deleted file mode 100644
index 376b3fc..0000000
--- a/
+++ /dev/null
@@ -1,200 +0,0 @@
-## Replicator
-### Overview
-Replication is a process that synchronizes two databases. All changes
-in one database (the source), inserts, updates, and deletes, are made
-to the second database (the target). The replication can be run on
-either the node containing the source (a *push* replication), or on the
-node containing the target database (a *pull* replication) or on a
-third node different from either the source or the target.
-#### Background
-This current version is based on the latest from CouchDB as partially
-documented on the CouchDB  [WIKI][1]. The first solid version that worked
-was implemented by [Adam Kocoloski][4] and subsequently picked up by [Filipe
-Manana][3], another CouchDB committer, who did two major reworkings. The
-first added the [replicator][2] db and the second some performance
-improvements and configurability to adjust factors that affect
-This was then ported and integrated with dbcore. The changes required
-were modest but there are notable differences in the use of the
-replicator db as noted below. These notes are intended to be the
-definitive resource for Cloudant users and will be kept in sync with
-code changes
-### Usage
-The following examples all assume the use of the `_replicate2` end
-point. Eventualy this will be replaced by `_replicate` but this allows
-us to support both the new and old replicator during the testing
-phase, .eg.:
-    curl -X POST -H 'content-type:application/json'
- -d
-    '{"source":","target":""}'

-In the remaining examples we'll just note the JSON
-bodies. `_replicate2` is really all there is to calling the
-replicator, all the parameters are in the JSON body. For example a
-simple pull replication (run from the target machine):
-    {"source":"",
-     "target":""}
-To create the target db also:
-    {"source":"",
-     "target":"",
-     "create_target": true}
-A continuous replication will stay running and as changes occur in the
-source, replicate them to the target. Under the covers it makes use of
-a continuous `_changes` feed:
-    {"source":"",
-     "target":"",
-     "continuous":true}
-A replication can be stopped by posting the same body but with a
-cancel property added:
-    {"source":"",
-     "target":"",
-     "continuous":true,
-     "cancel":true}
-When a continuous replication is run a replication id is returned,
-that provides an aditional method for cancelling the replication:
-    {"source":"",
-     "target":"",
-     "continuous":true}
-    {"ok":true,"_local_id":"0a81b645497e6270611ec3419767a584+continuous+create_target"}
-To cancel:
-    {"replication_id":
-    "0a81b645497e6270611ec3419767a584+continuous+create_target",
-    "cancel": true}
-These are the main use cases. Additionally there are filter functions
-supported, JS functions that control whether a doc is replicated. They
-work similar to views, .eg.:
-    {
-    "_id":"_design/foo",
-    "filters": {
-      "foo": function(doc, req) {
-                   if ( == req.query.key) {
-                      return true;
-                   } else {
-                      return false;
-                   }
-                }
-       }
-    }
-It's use is specified in the replication as follows:
-    {"source":" "", 
-     "target":"",
-     "filter":"foo/foo",
-     "query_params": {"key":"value"}
-    }
-Some applications might need to just replicate a few docs in which
-case `doc_ids` can be used:
-    {"source":"",
-     "target":"",
-     "doc_ids":["foo","bar","baz"]
-    }
-### Replicator DB
-A replicator db is a new way to manage replications better by storing
-them in a database. One just puts a JSON doc to the replicator db with
-a body the same as if is were posted to _replicate2
-    curl -X PUT 
- -d
-    '{"source":",
-      "target":""}'
-The doc might look something like:
-    "_id": "repl1",
-    "source":  "",
-    "target":  "", 
-    "create_target":  true,
-    "_replication_id":  "c0ebe9256695ff083347cbf95f93e280",
-    "_replication_state":  "triggered",
-    "_replication_state_time":  "2011-06-07T16:54:35+01:00"
-This replication now persists across server restarts and the
-replication can be cancelled by simply deleting the doc. The
-`_replication_state` will change to `completed` when a replication
-finishes, which may not happen in the case of a continuous
-replication. It can also change to `error` if it fails.
-### Advanced Configurations
-There are a few additional arguments that can be passed to a
-replication that govern it's performance. These are to be used with
-care as they can have signifificant impact on the system in a
-multi-tenant environment.
- * worker_processes - The default is 4. This controls how many
-   separate processes will read from the changes manager and write to
-   the target. A higher number can improve throughput.
- * worker_batch_size - The default is 500. This controls how many
-   documents are processed. After each batch a checkpoint is written
-   so this controls how frequently checkpointing occurs.
- * http_connections - The maximum number of http connections used per
-   replication. The default is 20.
- * connection_timeout - How long a connection can remain idle, the
-   default is 30000 (30s)
- * retries_per_request - When requests fail the number of times to
-   retry, the default is 10. There is a wait period between each
-   attempt, that begins with 0.25s and doubles on each iteration, with
-   a cap at 5 minutes.
- * socket_options - a list of options that can be used with the socket
-   connections. See the [erlang documentation][6] for details.
-### Differences from CouchDB
-CouchDB allows the replicator db to be shared by multiple users,
-controlling access via update handlers. It also allows the replicator
-db to be changed on the fly by setting a value in the config file. We
-support neither of these in dbcore. Each account has it's own
-replicator db, named _replicator, and that's it.
-Another difference to note is in specifying urls. Using the name of a
-local db works fine in a single CouchDB instance but makes less sense
-in a clustered environment. Though we try to determine a full url to
-use this doesn't work well in practice so it's a good rule of thumb to
-always use the full urls in both the source and target.
-### Overview of Code and Design
diff --git a/src/couch_replicator_sup.erl b/src/couch_replicator_sup.erl
index 848cb13..bfbc207 100644
--- a/src/couch_replicator_sup.erl
+++ b/src/couch_replicator_sup.erl
@@ -1,4 +1,3 @@
-% Copyright 2010 Cloudant
 % Licensed under the Apache License, Version 2.0 (the "License"); you may not
 % use this file except in compliance with the License. You may obtain a copy of

View raw message