couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ulises <ulises.cerv...@gmail.com>
Subject Re: another CouchDB evaluation Q
Date Tue, 25 Nov 2008 12:54:18 GMT
> To combat these problems I plan to have multiple write-slaves.
> Basically, writes will be load-balanced across the write slaves and
> then replicated from the slaves to my read master. If a delay between
> data being written and when it can be read is acceptable (and from
> your description it sounds like that's fine) then this is the best
> approach I see. You can add more write slaves as needed and they act
> as a sort of buffer. Set up a mechanism to periodically replicate the
> writes from the slaves to the master and then perform your analysis on
> the data in the master. Replication should be much faster than
> multiple small writes and not require as frequent compaction on the
> master. The slaves won't need compaction since you can just create a
> new database on a particular slave, add that new database to the load
> balancer pool, remove that slave's existing database from the pool,
> let that database finish replicating its changes, and delete it.

This is exactly the approach I've taken for a project that has been
live for about a month now. I basically have several mail scanning
servers each with their own couch which replicate to a master sofa
from which reads are done. Even though the load so far hasn't been
great on the couches (there's other things in between an email coming
in and actually making it to the couch) so far performance has been
great with insertion times remaining almost constant despite DB
growth. I also added a short-lived buffer that does a bulk_insert to
speed insertions a tad more.

In our case once a slave has replicate we can dump the daily DB and
start off fresh which makes couch a nice buffer indeed! :)

my 2p. here.

U

Mime
View raw message