From user-return-7095-apmail-couchdb-user-archive=couchdb.apache.org@couchdb.apache.org Mon Oct 26 16:13:34 2009 Return-Path: Delivered-To: apmail-couchdb-user-archive@www.apache.org Received: (qmail 10692 invoked from network); 26 Oct 2009 16:13:34 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 26 Oct 2009 16:13:34 -0000 Received: (qmail 21403 invoked by uid 500); 26 Oct 2009 16:13:33 -0000 Delivered-To: apmail-couchdb-user-archive@couchdb.apache.org Received: (qmail 21334 invoked by uid 500); 26 Oct 2009 16:13:33 -0000 Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@couchdb.apache.org Delivered-To: mailing list user@couchdb.apache.org Received: (qmail 21324 invoked by uid 99); 26 Oct 2009 16:13:33 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 26 Oct 2009 16:13:33 +0000 X-ASF-Spam-Status: No, hits=1.2 required=10.0 tests=FS_REPLICA,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of adam.kocoloski@gmail.com designates 74.125.78.27 as permitted sender) Received: from [74.125.78.27] (HELO ey-out-2122.google.com) (74.125.78.27) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 26 Oct 2009 16:13:23 +0000 Received: by ey-out-2122.google.com with SMTP id 22so2497700eye.41 for ; Mon, 26 Oct 2009 09:13:03 -0700 (PDT) Received: by 10.216.89.135 with SMTP id c7mr274802wef.62.1256573581393; Mon, 26 Oct 2009 09:13:01 -0700 (PDT) Received: from ?10.0.1.9? (c-66-31-20-188.hsd1.ma.comcast.net [66.31.20.188]) by mx.google.com with ESMTPS id p37sm9449614gvf.9.2009.10.26.09.12.58 (version=TLSv1/SSLv3 cipher=RC4-MD5); Mon, 26 Oct 2009 09:13:00 -0700 (PDT) Content-Type: text/plain; charset=us-ascii; format=flowed; delsp=yes Mime-Version: 1.0 (Apple Message framework v1076) Subject: Re: massive replication? From: Adam Kocoloski In-Reply-To: Date: Mon, 26 Oct 2009 12:12:56 -0400 Content-Transfer-Encoding: 7bit Message-Id: <05DD887F-C949-4A06-8A72-0ADBECEA05EB@apache.org> References: <4AE2028F.9030502@meetinghouse.net> <0C6A665F-53C0-41ED-9A8E-C00098B45174@apache.org> <4AE20CD6.7010500@meetinghouse.net> <4AE2247B.1010803@meetinghouse.net> <4AE5B610.90504@meetinghouse.net> <4AE5C134.7000108@meetinghouse.net> To: user@couchdb.apache.org X-Mailer: Apple Mail (2.1076) X-Virus-Checked: Checked by ClamAV on apache.org On Oct 26, 2009, at 11:35 AM, Chris Anderson wrote: > On Mon, Oct 26, 2009 at 8:33 AM, Miles Fidelman > wrote: >> Adam Kocoloski wrote: >>> >>> On Oct 26, 2009, at 10:45 AM, Miles Fidelman wrote: >>> >>>> The environment we're looking at is more of a mesh where >>>> connectivity is >>>> coming up and down - think mobile ad hoc networks. >>>> >>>> I like the idea of a replication bus, perhaps using something >>>> like spread >>>> (http://www.spread.org/) or spines (www.spines.org) as a multi- >>>> cast fabric. >>>> >>>> I'm thinking something like continuous replication - but where the >>>> updates are pushed to a multi-cast port rather than to a specific >>>> node, with >>>> each node subscribing to update feeds. >>>> >>>> Anybody have any thoughts on how that would play with the current >>>> replication and conflict resolution schemes? >>>> >>>> Miles Fidelman >>> >>> Hi Miles, this sounds like really cool stuff. Caveat: I have no >>> experience using Spread/Spines and very little experience with IP >>> multicasting, which I guess is what those tools try to reproduce in >>> internet-like environments. So bear with me if I ask stupid >>> questions. >>> >>> 1) Would the CouchDB servers be responsible for error detection and >>> correction? I imagine that complicates matters considerably, but it >>> wouldn't be impossible. >> >> Good question. I hadn't quite thought that far ahead. I think the >> basic >> answer is no (assume reliable multicast), but... some kind of healing >> mechanism would probably be required (see below). >>> >>> 2) When these CouchDB servers drop off for an extended period and >>> then >>> rejoin, how do they subscribe to the update feed from the >>> replication bus at >>> a particular sequence? This is really the key element of the >>> setup. When I >>> think of multicasting I think of video feeds and such, where if >>> you drop off >>> and rejoin you don't care about the old stuff you missed. That's >>> not the >>> case here. Does the bus store all this old feed data? >> >> Think of something like RSS, but with distributed infrastructure. >> A node would publish an update to a specific address (e.g., like >> publishing >> an RSS feed). >> >> All nodes would subscribe to the feed, and receive new messages in >> sequence. >> When picking up updates, you ask for everything after a particular >> sequence >> number. The update service maintains the data. > > The best candidate for an update service like this is probably a > CouchDB. Sounds that way to me, too, although that could be because CouchDB is the hammer I know really well. I'm still trying to figure out how multicast fits into the picture. I can see it really helping to reduce bandwidth and server load in a case where the nodes are all expected to be online 100% of the time, but if nodes are coming and going they're likely to be requesting feeds at different starting sequences much of the time. What's the win in that case? Best, Adam