incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Filipe David Manana <fdman...@apache.org>
Subject Re: Replication performance: pull vs. push
Date Wed, 02 Mar 2011 21:29:39 GMT
On Wed, Mar 2, 2011 at 9:09 PM, kowsik <kowsik@gmail.com> wrote:
> I have two couch instances in two different geographic locations (say
> A and B). I have two options to do bidirectional continuous
> replication.
>
> A.local -> B
> B.local -> A
>
> or
>
> A <- B.local
> B <- A.local
>
> The difference really, I think, is how the _changes feed works. In the
> first instance, the replication process on A (conceptually) monitors
> the local _changes feed and pushes data up to B (a request + a
> response for each change), while in the second case, the replication
> process monitors the _changes feed on B (across a network) and makes
> local changes.
>
> My gut tells me the second case will be better for performance since
> the TCP connection (for the _changes feed) from A to B persists and
> the docs are sent one way without making a request (and hence less
> data transmitted per change). Is my hypothesis correct? I haven't yet
> looked at the packets to understand the differences between the two
> scenarios. But maybe someone else has already.

Hi kowsik,

The _changes connection is only used to receive information about what
happened in each sequence number of the source. When doing a pull
replication, for each document/revision in a _changes line that is
missing in the target (local database, for a pull replication), the
replicator will get the document/revision in separate connection.

Pull replications used to be much faster because multiple processes
are spawned and each one fetches one document/revision, while for push
replications you had a single process sending documents to the target
(remote database).

This scenario changes with the new replicator (in trunk only) which
adds more parallelism and the difference between pull and push
replications is much shorter now.

Recently Wayne Conrad tried out this new replicator and saw much
better performance/behaviour compared to the old one:

http://mail-archives.apache.org/mod_mbox/couchdb-user/201103.mbox/%3CAANLkTimqLv5LzdeLBdLSmU1GOD0DO7zgPyqmK8LLnpYk@mail.gmail.com%3E

>
> Thanks,
>
> K.
> ---
> http://twitter.com/pcapr
> http://www.pcapr.net
> http://labs.mudynamics.com
>



-- 
Filipe David Manana,
fdmanana@gmail.com, fdmanana@apache.org

"Reasonable men adapt themselves to the world.
 Unreasonable men adapt the world to themselves.
 That's why all progress depends on unreasonable men."

Mime
View raw message