couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jens Alfke <>
Subject Re: A million databases
Date Tue, 25 Mar 2014 18:27:16 GMT

On Mar 25, 2014, at 12:41 AM, Suraj Kumar <> wrote:

> If there are a million "*.couch" files under var/lib/couchdb/, I'd expect the
> performance to be very poor / unpredictable since it now depends on the
> underlying file system's logic.

Do modern filesystems still have performance problems with large directories? I’m sure none
of them are representing directories as linear arrays of inodes anymore. I’ve been wondering
if this is just folk knowledge that’s no longer relevant.

> What is the "right" way to handle million users with need for isolated
> documents within each DB? How will replication solutions cope in
> distributing these million databases? 2 million replicating connections
> between two servers doesn't sound right.

The exploding number of replications is the main scalability problem, IMHO. Especially since
filtered replications don’t scale well.

One solution to this is the Couchbase Sync Gateway, which adds support for isolated and semi-isolated
documents within a single database without straying too far from the basic CouchDB model.
(Disclaimer: I’m the lead implementor of this, so I may be biased in its favor :)

View raw message