couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Dionne <dio...@dionne-associates.com>
Subject Re: CouchDB partitioning proposal
Date Sun, 19 Dec 2010 12:31:51 GMT

On Dec 18, 2010, at 8:00 PM, Klaus Trainer wrote:

> Hi guys!
> 
> 
> My two cents:
> 
> 
> If I had a few months to do some research in the area of Distributed
> Programming and CouchDB, I'd take the thread "How fast do CouchDB
> propagate changes to other nodes" on the user mailing list as an
> inspiration (which I've just read).
> 
> For instance, one could do some research about the challenges of having
> updates propagated in soft real time through a system of many loosely
> connected CouchDB nodes that get a lot of updates independently of each
> other. Maybe there's some room for optimizing CouchDB's replication, in
> particular for such scenarios.

Great suggestion. This is a challenging area, even within clusters of couchdb nodes that aren't
loosely coupled. There is information
that needs to be maintained globally, .ie. at all nodes for a healthy cluster and this needs
to be kept in sync. As Klaus mentioned
earlier BigCouch addresses a lot of the needs of distribution (it puts the C back in CouchDB),
and there are areas that need work, .eg. splitting/merging of partitions
dynamically while keeping the cluster up[1]. BigCouch has a well-defined architecture and
layered approach that makes exploration
and experimentation easier[1,2,3]. The inter-node communication component[2] was built to
be standalone and geared towards use with CouchDB.

Cheers,

Bob


[1] https://github.com/cloudant/mem3
[2] https://github.com/cloudant/rexi
[3] https://github.com/cloudant/fabric



> 
> At first, in order to find out about different possible tradeoffs, one
> would have to start comparing and evaluating different concepts.
> 
> For instance, one could find out about how replication things work, e.g.
> in CouchDB and in Riak. In terms of finding common ancestors of subtrees
> and detecting conflicts, there might be even a few things one could
> learn from Git...
> 
> 
> Anyway, you are welcomed to present new ideas! Or if not, some paper
> that gives an in-depth description of an existing feature of CouchDB
> (e.g. replication) would be great as well, as that provided insights for
> people who are not familiar with that particular codebase.
> 
> 
> Cheers,
> Klaus
> 
> 
> On Tue, 2010-12-14 at 22:54 +0000, Iago Abal wrote:
>> Hi all,
>> 
>> Well, to be more specific we are a group of classmates that have decided to
>> work on CouchDB as MSc coursework (Tiago might want to be brief...). We have
>> the task of study CouchDB until February and then, the idea is to spend 4-5
>> months working on a contribution for CouchDB. Our main problem seems to be
>> that wiki stuff is very out-of-date, when we read that CouchDB lacks feature
>> A and we decide to focus in this problem we finally find that it is already
>> solved. We have spent some time learn the very basic about CouchDB but we
>> are having troubles to properly define the project so we would appreciate
>> commentaries about what kind of contribution (related with distributed
>> systems topic) is of the interest of th CouchDB community.
>> 
>> Thanks in advance,
>> 
>> On Tue, Dec 14, 2010 at 10:03 PM, Klaus Trainer <klaus.trainer@web.de>wrote:
>> 
>>> Hi Tiago,
>>> 
>>> check out BigCouch: https://github.com/cloudant/bigcouch. Most of it has
>>> been done by the guys at Cloudant. They're building a scalable CouchDB
>>> hosting platform (https://cloudant.com), in which BigCouch is more or
>>> less the core of it. If you've any questions regarding Cloudant or
>>> BigCouch, you maybe can find some help in the #cloudant IRC room at
>>> Freenode.
>>> 
>>> For a (quick) introduction to BigCouch you can check out e.g.:
>>> 
>>> - "Dynamo and CouchDB Clusters":
>>> http://blog.cloudant.com/dynamo-and-couchdb-clusters
>>> - "Scaling Out with BigCouch"—Webcast: http://is.gd/iKLwM
>>> 
>>> Cheers,
>>> Klaus
>>> 
>>> 
>>> On Tue, 2010-12-14 at 21:34 +0000, Tiago Silva wrote:
>>>> Hi,
>>>> 
>>>> I want to contribute on CouchDB partitioning proposal (
>>>> http://wiki.apache.org/couchdb/Partitioning_proposal) and I would like
>>> to
>>>> know if anyone can help me to find the issues on this topic. Please tell
>>> me
>>>> what issues are being developed currently, the ones that are already
>>> closed
>>>> and what you suggest to start to develop now.
>>>> 
>>>> 
>>>> Thank you,
>>>> Tiago Silva
>>>> 
>>>> P.S. Please reply to all recipients of this mail message in order to
>>> better
>>>> filter this conversation in CouchDB dev mailing list.
>>> 
>>> 
>>> 
>> 
>> 
> 
> 


Mime
View raw message