couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Dionne <>
Subject Re: CouchDB partitioning proposal
Date Sun, 19 Dec 2010 12:31:51 GMT

On Dec 18, 2010, at 8:00 PM, Klaus Trainer wrote:

> Hi guys!
> My two cents:
> If I had a few months to do some research in the area of Distributed
> Programming and CouchDB, I'd take the thread "How fast do CouchDB
> propagate changes to other nodes" on the user mailing list as an
> inspiration (which I've just read).
> For instance, one could do some research about the challenges of having
> updates propagated in soft real time through a system of many loosely
> connected CouchDB nodes that get a lot of updates independently of each
> other. Maybe there's some room for optimizing CouchDB's replication, in
> particular for such scenarios.

Great suggestion. This is a challenging area, even within clusters of couchdb nodes that aren't
loosely coupled. There is information
that needs to be maintained globally, .ie. at all nodes for a healthy cluster and this needs
to be kept in sync. As Klaus mentioned
earlier BigCouch addresses a lot of the needs of distribution (it puts the C back in CouchDB),
and there are areas that need work, .eg. splitting/merging of partitions
dynamically while keeping the cluster up[1]. BigCouch has a well-defined architecture and
layered approach that makes exploration
and experimentation easier[1,2,3]. The inter-node communication component[2] was built to
be standalone and geared towards use with CouchDB.




> At first, in order to find out about different possible tradeoffs, one
> would have to start comparing and evaluating different concepts.
> For instance, one could find out about how replication things work, e.g.
> in CouchDB and in Riak. In terms of finding common ancestors of subtrees
> and detecting conflicts, there might be even a few things one could
> learn from Git...
> Anyway, you are welcomed to present new ideas! Or if not, some paper
> that gives an in-depth description of an existing feature of CouchDB
> (e.g. replication) would be great as well, as that provided insights for
> people who are not familiar with that particular codebase.
> Cheers,
> Klaus
> On Tue, 2010-12-14 at 22:54 +0000, Iago Abal wrote:
>> Hi all,
>> Well, to be more specific we are a group of classmates that have decided to
>> work on CouchDB as MSc coursework (Tiago might want to be brief...). We have
>> the task of study CouchDB until February and then, the idea is to spend 4-5
>> months working on a contribution for CouchDB. Our main problem seems to be
>> that wiki stuff is very out-of-date, when we read that CouchDB lacks feature
>> A and we decide to focus in this problem we finally find that it is already
>> solved. We have spent some time learn the very basic about CouchDB but we
>> are having troubles to properly define the project so we would appreciate
>> commentaries about what kind of contribution (related with distributed
>> systems topic) is of the interest of th CouchDB community.
>> Thanks in advance,
>> On Tue, Dec 14, 2010 at 10:03 PM, Klaus Trainer <>wrote:
>>> Hi Tiago,
>>> check out BigCouch: Most of it has
>>> been done by the guys at Cloudant. They're building a scalable CouchDB
>>> hosting platform (, in which BigCouch is more or
>>> less the core of it. If you've any questions regarding Cloudant or
>>> BigCouch, you maybe can find some help in the #cloudant IRC room at
>>> Freenode.
>>> For a (quick) introduction to BigCouch you can check out e.g.:
>>> - "Dynamo and CouchDB Clusters":
>>> - "Scaling Out with BigCouch"—Webcast:
>>> Cheers,
>>> Klaus
>>> On Tue, 2010-12-14 at 21:34 +0000, Tiago Silva wrote:
>>>> Hi,
>>>> I want to contribute on CouchDB partitioning proposal (
>>>> and I would like
>>> to
>>>> know if anyone can help me to find the issues on this topic. Please tell
>>> me
>>>> what issues are being developed currently, the ones that are already
>>> closed
>>>> and what you suggest to start to develop now.
>>>> Thank you,
>>>> Tiago Silva
>>>> P.S. Please reply to all recipients of this mail message in order to
>>> better
>>>> filter this conversation in CouchDB dev mailing list.

View raw message