Mailing-List: contact dev-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@couchdb.apache.org
Received-SPF: pass (nike.apache.org: domain of randall.leeds@gmail.com
 designates 209.85.219.168 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        b=nNsZwEgGKVafvUy6MWrzNAjo277c6hlQpQaQxzB48KKC8K0KWukLffD3N6rCTdd9/e
         WM0Cuykxyj4OwlB1qgLKNtZH7E2i+NJ94HMFygOiDrDk1lBrc85AIIc4Vfgv1a9fS4py
         1bkWOsUREjTk4KfcwjZLLyPezALtAQwsUyc3M=
MIME-Version: 1.0
In-Reply-To: <ec7a93a0905220927o695b68c0ha7b90e4f5ac01bd0@mail.gmail.com>
References: <a891e1bd0905210400n34981084t426fb3b55ca18ac6@mail.gmail.com>
	 <067AA5E4-0E5F-46C7-85EE-FC9CBCF99490@apache.org>
	 <a891e1bd0905212030g67001a1akd02bf85a6aaa4ad9@mail.gmail.com>
	 <e2111bbb0905212134p2503e841o5cad5637f4b76d31@mail.gmail.com>
	 <ec7a93a0905220927o695b68c0ha7b90e4f5ac01bd0@mail.gmail.com>
Date: Fri, 22 May 2009 14:30:29 -0400
Message-ID: <ec7a93a0905221130l4b60b7caq7b2a54816c5808ec@mail.gmail.com>
Subject: Re: reiterating transactions vs. replication
From: Randall Leeds <randall.leeds@gmail.com>
To: dev@couchdb.apache.org
Content-Type: multipart/alternative; boundary=0016367b6af2836530046a847419

--0016367b6af2836530046a847419
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On Fri, May 22, 2009 at 12:27, Randall Leeds <randall.leeds@gmail.com>wrote:
>
>
> Since I do like the component model, I'm planning to set up a github
> project to play with some consensus protocols and overlay networks in
> Erlang. Hopefully once I start doing that I'll start to see the places that
> CouchDB can hook into it and get a nice, clean, flexible API. I see the
> problem broken into several tiers.
>
> Transactional Bulk Docs (this is the wishlist and challenge, but has to
> rest on the below)
> Sharding/Replication (_seq consensus / possibly consistent hashing or other
> distributed, deterministic data structure mapping BTree nodes to servers
> [2])
> Communication (either Erlang or a tcp with pluggable overlay-network for
> routing)
>

A revised break-down should be something like:

Transactional Bulk-Docs
Single-Doc Multi-Replica Transactions
Replication / Sharding
Network

Example:

Transactional Bulk-Docs (Server pre-prepares itself as leader for a special
bulk round)
Single-Doc Multi-Replica Transactions (Simple consensus. Special leader for
bulk case. Pre-determined leader normally.)
Replication / Sharding (Any sort of load-balancing, slicing, or static
configuration)
Network (Chord and derivatives (Scalaris uses Chord #), Tapestry, Pastry,
etc)

I think with the right configurations and components transactional bulk-docs
are just a special case of single-doc transactions. For example, in case the
single-doc layer optimizes for less communication rounds by pre-selecting
leaders on a rotating basis a bulk transaction just involves revoking all
nodes for a sequence number consensus round and using an extra round trip to
"take over" the leader position. Then all nodes holding replicas of all
documents involved would have to participate in this new round (or at least
a majority of replicas). Having 'atomic=false' could skip this expense and
make a best-effort serial execution of the updates and fail on conflict.

Just trying to keep the conversation rolling. But I understand we have to
hit the code soon if this really stands to go somewhere.

--0016367b6af2836530046a847419--