couchdb-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Couchdb Wiki] Update of "BreakingChangesUpdateTrunkTo0Dot9" by JanLehnardt
Date Sun, 15 Mar 2009 17:57:58 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Couchdb Wiki" for change notification.

The following page has been changed by JanLehnardt:
http://wiki.apache.org/couchdb/BreakingChangesUpdateTrunkTo0Dot9

The comment on the change is:
add trunk to 0.9 update page

New page:
The latest CouchDB update brings a mature replication model for distributed updates. This
involved quite a lot of changes to the internal handling of MVCC tokens. The result is what
we think will be a stable and scalable model for distributed updates. It also means that both
the replication API and the database file format have changed at the same time.

Normally when the database file format changes upgrading can be accomplished by launching
two copies of CouchDB on two different ports and replicating between them. In this case that
won't work as the replication API has changed.

I've written a script to do the replication for you. It's linked there and embedded below.
It's not perfectly awesome (see caveats later) but it does the trick and won't blow up on
databases with lots of docs. Nor does it fill up your filesystem with intermediate data.

== Usage ==
First you need the latest CouchRest Ruby gem (0.2.2). I never met a packaging system I didn't
hate, so I won't even suggest that you attempt to install CouchRest from a remote gem server.
Instead do this:

{{{
git clone git://github.com/jchris/couchrest.git
cd couchrest
gem build couchrest.gemspec 
sudo gem install couchrest-*.gem
}}}

Now that you have the latest CouchRest, you can download the script, modify it to run in your
environment, and start it up. It does give some output every 100 docs, so you won't be totally
at a loss to what's happening.

It assumes that none of the databases on your `old_couch` exist on your `new_couch`. If they
do exist it will skip them.

It also assumes that none of your individual attachments are larger than the memory you can
dedicate to the Ruby runtime. Coding a streaming attachment solution would be a fair amount
more work. If you need it patches are welcome but I think you might be better off with a different
approach. My script batches updates into blocks of 100 docs.

If you have giant attachments, I'd do the docs one at a time, and stream each attachment individually.
It'd probably be easier to shell out to curl from Ruby than to try to code it as a Ruby `Net::HTTP`
operation.

Dueling Couches
To run CouchDB in two ports at once, you should run them in separate directories. The easiest
way to do this from the source package is with the make dev target. You'll want to checkout
CouchDB trunk in one directory, like so:

{{{
svn co https://svn.apache.org/repos/asf/couchdb/trunk
}}}

And then there's a handy tag of the last time the old file format was available, so check
it out in another directory:

{{{
svn co https://svn.apache.org/repos/asf/couchdb/tags/bulk_transactions
}}}

Once you have make dev completed in both checkouts, copy your existing old-format databases
(the `.couch` files) to the `tmp/lib` directory of the bulk_transactions checkout. When you
run it with utils/run you should be able to browse those databases in Futon.

Now, edit the trunk CouchDB's configuration so that it runs on port 5985 instead of 5984.
This you can do by changing the `etc/couchdb/local_dev.ini` file that was created by `make
dev`. Once that's done you can launch trunk CouchDB with `utils/run`

Mime
View raw message