Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@couchdb.apache.org
Received-SPF: pass (athena.apache.org: domain of binary42@gmail.com designates
 209.85.218.52 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=date:from:to:message-id:in-reply-to:references:subject:x-mailer
         :mime-version:content-type;
        b=ZI6YgMOF7jU8ox4G4YU38Dw16zkprDESB3OEo1Bin0H9IQIR5yrpvti/KCscK2LCrR
         OAnBxnORNFApMj3+ckfe3bnnHHyypg4Enl0C6OOYyFPwi3ObDQoulD+fDS/XdQkD5Kvz
         qqphy9eqy+ycQwUZtp32/tROzD81e4JgBo964=
Date: Thu, 26 May 2011 11:21:43 -0400
From: Brian Mitchell <binary42@gmail.com>
To: user@couchdb.apache.org, glenn.bech@gmail.com
Message-ID: <4805811FE46943C0ADA56EA0BE7D9413@gmail.com>
In-Reply-To: <BANLkTinvekctxMpfYiHa=EdCjJgV2-ToAg@mail.gmail.com>
References: <BANLkTinvekctxMpfYiHa=EdCjJgV2-ToAg@mail.gmail.com>
Subject: Re: Limit on the number of databases?
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary="4dde7007_5c482a97_2115"

--4dde7007_5c482a97_2115
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline


On Thursday, May 26, 2011 at 6:22 AM, Glenn Bech wrote:

> Hi,
> 
> I just want to ask if there are limits on the number of databases in Couch.
> I am playing around with embeded Couch on Android and are thinking in the
> line of having
> one database per user, and use replication to push data from the client to
> the server. This will provde for an Excellent "offline" user experience.
> 
> This will of course not work if Couch does not handle unlimited datbases
> very well performance- or otherwise.
> 
> Does this sound like a feasable design solution?
> 
> Regards,
> 
> Glenn
 I've done some testing and there are a couple things to keep in mind.

First of all, CouchDB relies directly on the scalability of your filesystem. Having one database in CouchDB means you also have at least one file for each of those. Since CouchDB currently stores them all in one directory, you'll need to make sure you select a filesystem that can handle your expected scale appropriately (many filesystems should be fine in the millions of files level, but characteristics can differ so do test this). 

Another problem, one which I don't have an immediate answer for is backup. While you could claim replication is enough for this, I'd say it isn't. The event you need backups for also cover events like maliciously destroyed or manipulated data or simply the existence of bugs. I'd rather not trust my data never get screwed up. by the code that accesses it. Many backup systems are designed around a small number of files. Being able to rollback to a point in time with millions of files could be an extremely painful process. (I have ideas on how to solve this but it's still not an easy problem.)

Last but not least, consider the number of active databases you'll need at any single time. This can be split across many machines of course but it still adds up quickly. Open file descriptors are great but not if you have to close and then reopen them all the time. A carefully tuned VM can manage many thousands w/o a problem but I wouldn't push this too much higher. So if you have 15 machines and 30k active users for any single 1 minute window, that would be 2k files open and active per machine.

Brian. 
--4dde7007_5c482a97_2115--