incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Randall Leeds <randall.le...@gmail.com>
Subject Re: Moving Partitions
Date Tue, 03 May 2011 14:37:15 GMT
On Mon, May 2, 2011 at 06:43, Stefan Podkowinski <spodxx@gmail.com> wrote:
> Hello Couch Users
>
> As I'm about to develop a deployment strategy for my current company,
> theres one aspect left that keeps me puzzled. While couch seems to be
> pretty disk hungry I need a way to scale in terms of disk space and
> decided to setup oversharding right from the start. I plan to create a
> couple of logical partitions on a single server that I can move later
> to another server to free disk space. Propably using lounge for that.
>

The way Lounge works (and BigCouch also, because these days they use
the same scheme, last I checked) is to proxy the database name for the
logical pieces of your data.
That is, instead of having one database 'stuff' you have databases
called shards/00000000-1cffffff/stuff, each with their own hex range
representation of the data contained therein. A request to 'stuff'
composes the results from the calls to the various
shards/<hex-range>/stuff databases.

> What I'm wondering though is how to move the data to a new node in
> that case. E.g. lets say I have node A with 16 logical partitions. Now
> I like to move 8 of them to a new node B. How can I move the data for
> only those 8 partitions to the new node? I can't just reimport the
> whole data as that would take too much time. Is that even possible to
> split an existing node that way?
>

With this setup, moving a shard is a straightforward, normal replication.
In theory you could also perform filtered replication to split a shard
into two pieces with two target databases.
I haven't had to do this in practice yet with Lounge, but Cloudant may
have done this in production with BigCouch.
I don't know whether BigCouch automates any of these tasks. Lounge
currently does not, though I have a few scripts that help.

Mime
View raw message