couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adam Kocoloski <kocol...@apache.org>
Subject Re: A couple of noob questions.
Date Wed, 12 Aug 2009 14:22:41 GMT
On Aug 12, 2009, at 7:52 AM, Tim Uckun wrote:

> Has anybody done any benchmarks on the replication speed of couchdb?
> If I have four servers each with couchdb running on them along with a
> client application that talks to it and one client makes an entry how
> long will it take for the other three to get the entry?

In CouchDB 0.9.x (0.9.1 is the current release) you have to trigger  
the replication manually.  You can write a script that listens for  
update_notifications and POSTs to _replicate.  So that's once source  
of lag.

Replication basically loops through the following 4 steps:

1. ask the source for new updates since the last replication
2. ask the target which of those updates it does not have
3. get the missing updates on the source
4. write them to the target

In most cases, 2 of these will be local requests and two will be  
remote HTTP requests, so the latency between the source and target  
enters into the equation twice.

Will all that said, its certainly possible to configure things so that  
latencies are O(100ms) for single document updates.  I'm afraid there  
isn't a great way to measure this in 0.9.x.

In the upcoming release CouchDB will have some facility for continuous  
replication, which takes the burden off of you to trigger on update  
and reduces the lag for updates to show up on the target.  Also, each  
DB will have a _changes feed that you can listen to.  A script that  
listens to _changes on both source and target could do a reasonable  
calculation of the replication lag.

> If each client makes a thousand entries in a 15 minute period will
> there be a huge backlog built up waiting to replicate?

It depends on how you're triggering the replication.  If you're  
triggering it on update (or using continuous replication in the  
upcoming release) there shouldn't be any backlog.

In CouchDB 0.9.x push replication (local source, remote target)  
generally seems to run at about 80-100 docs/second for me using O(1KB)  
documents and no attachments.  Pull replication (remote source, local  
target) is significantly slower because it replicates documents one at  
a time.  The trunk code is faster than either of these.

> Also is it possible to put files in the couchdb? Specifically media
> files like wav.

Yes, absolutely.  We do this using document attachments -- PUT /dbname/ 
docname/media.wav

> Will they replicate as well?

Yes, although replicating attachments uses a lot more memory than it  
should.  We're working on that.  I'd stuck to a couple of MBs or less  
per document for now.

Best, Adam


Mime
View raw message