incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Randall Leeds <randall.le...@gmail.com>
Subject Re: Copying view file between replicated servers
Date Mon, 24 Jan 2011 11:26:58 GMT
On Mon, Jan 24, 2011 at 01:01, Paul Hirst <paul.hirst@sophos.com> wrote:
> I currently have a setup with two servers running couch replicating a
> large database between them. One of them is 'live' in the sense that is
> receives all the reads and writes. The other is basically a backup copy.
>

Views are updated incrementally and the sequence number of the
database is stored inside the index. However, on recent versions of
CouchDB two updates to the same document in rapid succession may be
collapsed into one replicated change in order to expedite replication.
It is generally not safe to assume that, even it receives no outside
writes, your target will have the exact same update history as your
source.

> I just created a new design document with a new view and I just queried
> the new view on the backup server in order to trigger an index build.
> It's going to take a few hours to build. I don't really want that load
> on the live server since it will slow it down too much. I was wondering
> if, when the index build finishes, I can copy the view file from the
> backup server to the live server. Will that work or are the view files
> in some sense server/database specific?
>

As a consequence of the strategy mentioned above, this approach would
work if your update sequence were *identical* on the two databases. I
*WOULD NOT RECOMMEND* doing this, though. It would certainly be
unsupported behavior if you got lucky.

I have two ideas if you need an alternative, but it depends on what
you're trying to avoid.
If you cannot deal with waiting for the new index to generate before
querying it, create the new views in a separate design doc. Query that
and wait for it to build. Once it has finished, rename the design
document (update the old one) and your views should be "pre-indexed".

If you cannot deal with the load generated by indexing itself, you
could create a remote query server. Be sure that the CouchDB user can
SSH without password and add ssh to the beginning of your query server
command.

For example, if default.ini contains:
[query_servers]
javascript = /usr/local/bin/couchjs /usr/local/share/couchdb/server/main.js

You could add the following to local.ini:
[query_servers]
javascript = ssh backupmachine /usr/local/bin/couchjs
/usr/local/share/couchdb/server/main.js

To be safe, try this on other machines first or call the query server
"javascript_ssh" or some such thing and create a design doc with
"javascript_ssh" as the language for a view. Unfortunately, it looks
as though the hooks for CouchDB to pick up changes to the query server
setup aren't there yet, so you can't do this on the fly.

However, if you're still with me and feeling brave, I think you could
probably *replace* the couchjs script with a bash script that pipes
to/from couchjs on the backup machine over SSH and then just *kill*
any running couchjs processes. CouchDB will restart them and get your
new, remote indexing script.

If all if this makes perfect sense, you can go ahead and give it a
shot. If it sounds terrifying, lets talk about it or catch me on IRC
(tilgovi). This is the first time I've recommended anything like this
be tried, so it probably deserves some close inspection before blindly
listening to a word :).

Cheers,
Randall

Mime
View raw message