couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Davis <>
Subject Re: dbcopy - A faster alternative to replication
Date Sat, 22 Jan 2011 00:14:06 GMT
On Fri, Jan 21, 2011 at 6:58 PM, Chris Stockton
<> wrote:
> Hello,
> After some discussion about user migration and issues with 1GIG+ sized
> databases, it seems we will not be able to use replication for copying
> databases. We have seen the speed offered by rsync/cp in comparison to
> replication in couchdb. Our thoughts currently lie in a external
> handler to make this a bit easier, rsync / ssh user permissions across
> our server draws away from some of our security methodologies but it
> is workable.
> My question is could(or would rather) a efficient database copy ever
> be implemented in couch similar to the way replication API works? One
> that optimized both local server and remote server database copying.
> If no future plans exist what would be the possible implementation
> details, roadmap/business rules holding such a feature back? Does
> anyone else have any sort of need or desire for faster, built in
> database copying?
> Kind Regards,
> -Chris

Not sure on your specific use case, but its not uncommon for people to
seed replication for large databases with rsync/scp and then revert to
normal replication once things have been initialized. Another thing to
keep in mind is that you'll get slowdowns if you don't compact very
often as replication will have to scan pass more dead data.

In terms of an rsync/cp feature, that's exactly what replication is.
There's a new version of the replicator that's being tested before we
push it to trunk, but there are people working on optimizing that part
of the code. There has been random discussions in various places on
the problem when initiating the first replication between a large db
and an empty destintation, but I haven't seen anyone actually try and
write that feature, but it is one that would be seriously considered.

View raw message