incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Miles Fidelman <mfidel...@meetinghouse.net>
Subject Re: massive replication?
Date Mon, 26 Oct 2009 17:05:36 GMT
Adam Kocoloski wrote:
> On Oct 26, 2009, at 11:35 AM, Chris Anderson wrote:
>
>>
>>>>
>>>> 2) When these CouchDB servers drop off for an extended period and then
>>>> rejoin, how do they subscribe to the update feed from the 
>>>> replication bus at
>>>> a particular sequence?  This is really the key element of the 
>>>> setup.  When I
>>>> think of multicasting I think of video feeds and such, where if you 
>>>> drop off
>>>> and rejoin you don't care about the old stuff you missed.  That's 
>>>> not the
>>>> case here.  Does the bus store all this old feed data?
>>>
>>> Think of something like RSS, but with distributed infrastructure.
>>> A node would publish an update to a specific address (e.g., like 
>>> publishing
>>> an RSS feed).
>>>
>>> All nodes would subscribe to the feed, and receive new messages in 
>>> sequence.
>>>  When picking up updates, you ask for everything after a particular 
>>> sequence
>>> number.  The update service maintains the data.
>>
>> The best candidate for an update service like this is probably a 
>> CouchDB.
>
> Sounds that way to me, too, although that could be because CouchDB is 
> the hammer I know really well.
>
> I'm still trying to figure out how multicast fits into the picture.  I 
> can see it really helping to reduce bandwidth and server load in a 
> case where the nodes are all expected to be online 100% of the time, 
> but if nodes are coming and going they're likely to be requesting 
> feeds at different starting sequences much of the time.  What's the 
> win in that case?
>
Doesn't seem that way to me.  At the very least, for a fully distributed 
design (which is what we're seeking), this would require a backbone of 
multiple CouchDB instances plus a management infrastructure of some sort.

What I'm looking for is a way to avoid:

1. any kind of central node
2. the need to manage an unbounded number of 1-1 links between nodes

That requires some kind of many-many protocol that takes care of moving 
messages around.

Miles


-- 
In theory, there is no difference between theory and practice.
In practice, there is.   .... Yogi Berra



Mime
View raw message