couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Anderson <>
Subject Re: continuous replication API discussion
Date Wed, 12 Aug 2009 04:41:52 GMT
On Tue, Aug 11, 2009 at 9:03 PM, Adam Kocoloski<> wrote:
> On Aug 11, 2009, at 5:57 PM, Chris Anderson wrote:
>>>> 4. How can we configure replications that survive a server restart?
>>> I think I'd like to see a replication config database, and a constant
>>> replicator config process that reads the config info at server start and
>>> starts the continuous replications.  It monitors the config db for
>>> changes
>>> and as new settings are written spawns, restarts or shuts down
>>> replication
>>> tasks.
>> I think this is a great idea. The Futon interface for configuring this
>> will probably take a lot of the mystery out of replication for our
>> users.
> Sure, we were just using a [replication] block in an .ini file for this
> purpose internally for a while now, but a DB is obviously much more
> flexible.  couch_rep_sup can easily monitor the config DB for changes and
> take the appropriate action(s).
> The next question I have is whether there ought to be any correspondence
> between the doc in the configuration DB for a particular replication and the
> _local doc in the DB itself that we use to record replication history.  Some
> thoughts:
> * we don't expose a view of _local docs, so currently users just have to
> guess the right ID to see the history.

I think a view of _local docs would be ideal. I've been wishing for
this or a long time, glad to see we're getting close to it.

> * we could automatically add a field to the config document that identifies
> the corresponding _local doc ID.  We'd have to use something like the new
> _update handler to validate this field on every update, as a change in
> authentication information means a change in the corresponding _local doc
> ID.

I think trying to keep them in sync is a smell. The replication
manager can perhaps just use a group view of replication history to
list all the potential replication targets.

> * replication history can get updated very frequently, so for users' sanity
> we probably don't want to actually save the history in the configuration
> document.


> A thought -- what if one-off POSTs to /_replicate automatically saved a
> config document as well, but with a "continuous":false field?  Then it would
> be easy to view all the replications that ever originated on the server, and
> admins could make certain ones continuous just by flipping a bit.

+1 that makes the replication manager easier to build, and it give it
bookmarks for triggered replication.

But as you can see there are tensions between handling replication
history with a view on local docs, vs storing a configuration

What kind of _local doc artifacts does continuous replication leave in
the current implementation - maybe a _local view can give us
everything we need.

> And most importantly, what do we call this DB? :-P  Couch has always used
> _underscored reserved names, but now we've got a "users" DB.  Best,
> Adam

"_users" as a database name would be fine by me, actually it's
probably a good idea. A "_config" db starts to smell like it should
just be in the ini. But I think it's worth it to have a JSON
replication config API, and I think if we had a _config database it'd
be a great place to set configuration.


Chris Anderson

View raw message