Mailing-List: contact dev-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@couchdb.apache.org
Received-SPF: pass (athena.apache.org: domain of klimpong@gmail.com designates
 209.85.161.52 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:sender:reply-to:in-reply-to:references:date
         :x-google-sender-auth:message-id:subject:from:to:content-type
         :content-transfer-encoding;
        b=cre4MzvoqwEpzQwYp4yvIPJu9cwwJdwNf66ckJAZy/sRx13UWne5zKiTc2/gShjCOM
         t2g+WqrvfJnOZxdxqhVDNdw+mkKF4UxC5/oVHSZ+al6BsYDcJ+F9NmFrOTaplBSQbv0q
         Wlg1qAR/7AwDm9Y+Byq0MQKetnkDNRfMsSDN0=
MIME-Version: 1.0
Sender: klimpong@gmail.com
Reply-To: till@php.net
In-Reply-To: <AANLkTikJbjd4oXCkeRoQk3AXVMt0JP9e7cuA03Y35v3j@mail.gmail.com>
References: <AANLkTinTBrdI4rqrQGEZyMBq4bsogLD_ShkcI6h_FvuX@mail.gmail.com>
	 <AANLkTikJbjd4oXCkeRoQk3AXVMt0JP9e7cuA03Y35v3j@mail.gmail.com>
Date: Mon, 24 May 2010 18:53:40 +0200
Message-ID: <AANLkTim4C1mRgYsL5U0_pdQawJLlGRf68dqw9TsACy0L@mail.gmail.com>
Subject: Re: scalability of couchdb
From: till <till@php.net>
To: dev@couchdb.apache.org
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: quoted-printable

On Mon, May 24, 2010 at 9:25 AM, Matteo Redaelli
<matteo.redaelli@gmail.com> wrote:
> Hello
>
> I currectly use couchdb as repository for my the Ebot project (
> http://www.redaelli.org/matteo-blog/projects/ebot/). My db is getting mor=
e
> and more bigger..
>
> =A0 rw-r-r=97 1 couchdb =A081099821870 2010-05-23 ebot.couch
> =A0 rw-r-r=97 1 couchdb =A0392589468 2010-05-23 =A0.ebot_design
>
> For better scalability it would be better that
>
> 1) couchdb wrote data not only to one single file but to different files =
to
> DIFFERENT directories: in this way it would be easier to add new devices,
> file systems ...
> 2) couchdb distributed (non only replicated) data among several instances=
.
> like Riak and other NOSQL databases
>
> Do you think that these 2 features could be implemented in the next near
> future? or are they out of the scope of couchdb project?
>
> Thanks in advance

Hi Matteo,

you should try CouchDB-Lounge for your setup. It's not yet as smooth
as riak or cassandra do partitioning (and auto-balancing, etc.) but it
should get you started as well.

Further reading:
http://tilgovi.github.com/couchdb-lounge/
http://groups.google.com/group/couchdb-lounge
http://till.klampaeckel.de/blog/archives/84-A-toolchain-for-CouchDB-Lounge.=
html

CouchDB-Lounge is nginx, a twistd daemon and several CouchDB servers
which can all run on the same server, of course. It'll help you
utilize all the resources until you outgrow your current server and
then it's trivial to move a couple of your shards to another one and
spread it out.

CouchDB-Lounge currently doesn't do auto-balancing which is useful if
you want to add another server to your cluster. Instead the proposal
is to overshard (run more CouchDB instances than you need on the same
server) or add another lounge to the cluster and partition "behind"
it.

If you're running into space issues right now it helps to move the
views (.ebot_*) to another disk and configure view_index_dir in your
local.ini.

Hope that helps!

Cheers,
Till