Mailing-List: contact dev-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@couchdb.apache.org
Received-SPF: pass (athena.apache.org: domain of jchris@gmail.com designates
 209.85.221.21 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:sender:in-reply-to:references:date
         :x-google-sender-auth:message-id:subject:from:to:content-type
         :content-transfer-encoding;
        b=nCfrupIVDjdBMIqMbxdTcmAZzsM3b+DbwE9ZJHOfLGf8jXXQKyQfDDBPd4r8ZKOylv
         AHryN9ZYJZ3k1M3CZFAjULJYBI6yS6OgTbJKSPQ1yhaaSFScndDlhtgXst/K+uNNkIvl
         PBYk/svod0aJxCALedXR2jqNQCSyscnGOjNc4=
MIME-Version: 1.0
Sender: jchris@gmail.com
In-Reply-To: <e2111bbb0902181524u30933587pd3a07750f9f23b92@mail.gmail.com>
References: <e282921e0902181509g42cbefa1yf42fce27278b2649@mail.gmail.com>
	 <e2111bbb0902181524u30933587pd3a07750f9f23b92@mail.gmail.com>
Date: Wed, 18 Feb 2009 15:40:57 -0800
Message-ID: <e282921e0902181540m7ec4223eieee9f3ed5e85e24d@mail.gmail.com>
Subject: Re: Peer Commit
From: Chris Anderson <jchris@apache.org>
To: dev@couchdb.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

On Wed, Feb 18, 2009 at 3:24 PM, Paul Davis <paul.joseph.davis@gmail.com> wrote:
> On Wed, Feb 18, 2009 at 6:09 PM, Chris Anderson <jchris@apache.org> wrote:
>> I don't know the literature, but I have an idea for a shared
>> data-space across peer nodes (not in a cluster).
>>
>> A replication helper, which tracks all known replicas, and their last
>> successfully replicated sequence num can provide to the user a list of
>> remote nodes that have knowledge of each update.
>>
>> I think this works better with POST than GET replication... but by
>> logging GET replications we could have a rough idea as well.
>>
>> If we track sequence id progress of all known replicas, we know with
>> what confidence local updates have been propagated to the network.
>>
>> We can add secure messaging eventually, but for starters even a good
>> public data flow can get us a long way. The next step here is
>> peer-discovery.
>>
>> It makes sense to me that distributed applications should expose their
>> replication state to the end user. Peer commit, or maybe eventual peer
>> commit, on an individual document update basis, does not make any
>> guarantees beyond the most basic, but still provides the ability to
>> publish content to a wide audience via a distributed network.
>>
>> Example: When a user in a p2p mesh can see that N remote nodes are
>> keeping roughly up to date with her blog, it gives the proper
>> granularity of feedback. In the CouchDB Web, we don't count page hits,
>> we count replication streams.
>>
>> I think it's the simplest metaphor, and avoids false abstractions like
>> bulk commits and order guarantees.
>>
>> Chris
>>
>> --
>> Chris Anderson
>> http://jchris.mfdz.com
>>
>
> Chris,
>
> Two questions.
>
> What happens when we move beyond a star topology into a true mesh. Ie,
> You've replicated Jan's blog, and I replicate from you?

I guess this provides only a minimum guarantee. Eg, a user knows that
nodes A, B, and C have the update. Node D could have the update
without the user knowing.

>
> Also, the number of replicas is essentially unbounded, assuming that
> we were even just keeping a list of (UUID, update_seq) tuples, we'll
> eventually have some headaches in terms of storage (depending on the
> answer to the first question). Though there may be other ways to track
> this information with bloom filters or some such.
>

I wonder if the current _local replication records could support this view?


-- 
Chris Anderson
http://jchris.mfdz.com