couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Anderson <jch...@apache.org>
Subject Re: Partitioned Clusters
Date Fri, 20 Feb 2009 19:28:42 GMT
On Fri, Feb 20, 2009 at 10:55 AM, Stefan Karpinski
<stefan.karpinski@gmail.com> wrote:
> Hi, I thought I'd introduce myself since I'm new here on the couchdb
> list. I'm Stefan Karpinski. I've worked in the Monitoring Group at
> Akamai, Operations R&D at Citrix Online, and I'm nearly done with a
> PhD in computer networking at the moment. So I guess I've thought
> about this kind of stuff a bit ;-)

Glad to have you with us. :)

>
> I'm curious what the motivation behind a tree topology is. Not that
> it's not a viable approach, just why that and not a load-balancer in
> front of a bunch of "leaves" with lateral propagation between the
> leaves? Why should the load-balancing/proxying/caching node even be
> running couchdb?

The reason to write the proxies as Erlang, is that they can avoid the
JSON and HTTP overhead until the final stage, as well as use Erlang's
inter-node communication and process management mojo.

The tree structure also provides a nice mapping onto the existing
reduce implementation. Inner nodes can store the reduction values for
their leaf nodes and run the reduce function to come up with total
values.

>
> One reason I can see for a tree topology would be the hierarchical
> cache effect. But that would likely only make sense in certain
> circumstances. Being able to configure the topology to meet various
> needs, rather than enforcing one particular topology makes more sense
> to me overall.

I agree - as Ben points out the flat topology is just a special case
of the tree (and would probably be ideal for anything less than
hundreds of nodes).

I'm not an expert on cluster layout, but the tree structure appeals to
me mostly because changes to subtrees don't need to be propagated to
the cluster root.

That said, there's *plenty* that can be done with HTTP proxies (and
probably implemented more quickly) so it's probably the best way to
prototype any of these implementations.

Chris

-- 
Chris Anderson
http://jchris.mfdz.com

Mime
View raw message