Mailing-List: contact dev-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@couchdb.apache.org
Received-SPF: pass (athena.apache.org: domain of jchris@gmail.com designates
 209.85.221.21 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:sender:in-reply-to:references:date
         :x-google-sender-auth:message-id:subject:from:to:content-type
         :content-transfer-encoding;
        b=wzKYv3PtH2EikCVn15rdCpZ+C39HFnKXJCJrlsqkO/mL+P4A7w1LtGo2WhSHJ707rr
         QMw8VGHcm3M/EClx2B04hd2TaIVjnTVxJiyX1Jm2ZzzNgvEHcUbiDqAnwC+l1p65LW9s
         NFu3ReneUuBzmU78mEuvNn5eq/DgXZFj6HoSs=
MIME-Version: 1.0
Sender: jchris@gmail.com
In-Reply-To: <D89DB717-DAF3-4120-A36B-85EE489A0388@apache.org>
References: <BC4993A1-5DE9-43B6-AA57-4D2362EA72CE@gmail.com>
	 <988C8AAF-E151-40FB-9E1A-000876FE3489@gmail.com>
	 <182D5B6E-D179-470A-8638-B54E3DEF2747@pobox.com>
	 <11E11144-004D-45B8-A503-88FD471953D7@apache.org>
	 <9C8B5F07-856F-495D-AD91-FCA5AB5E31FF@pobox.com>
	 <4E507D2E-88F9-4591-B721-F4343ACA9A9E@apache.org>
	 <393666B7-8444-4D23-A2BA-AD59652A96AE@sauria.com>
	 <BF0132C0-ACC7-4A5C-BE68-4E41AE20373C@apache.org>
	 <0D17D25F-7E88-4F19-96A9-62FC81E2DFC5@pobox.com>
	 <D89DB717-DAF3-4120-A36B-85EE489A0388@apache.org>
Date: Thu, 5 Feb 2009 11:50:49 -0800
Message-ID: <e282921e0902051150g2aeda88fr2789f9f19468b30c@mail.gmail.com>
Subject: Re: Transactional _bulk_docs
From: Chris Anderson <jchris@apache.org>
To: dev@couchdb.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

On Thu, Feb 5, 2009 at 5:34 AM, Damien Katz <damien@apache.org> wrote:
> We are going to discuss this on the ML. I was waiting until I got the patch
> work to talk about all the implications and how we'd set the flags and modes
> of operation and all the implications. The code is going to get more
> powerful, the plan is for the feature to go away, not the capability. If we
> decided the feature was too important, we'll put it back. But as it stands,
> the changes to the code that I'm making now all need to be made regardless
> if we change the feature or not.

I agree that we should discuss this on the mailing list, and take a
formal vote when we're ready to. I'm glad we're talking about the
patch, but I can see why Damien would rather finish writing it before
we take it apart on here.

I opened my response to the this thread by asking for interested
parties to discuss how one would implement the bulk_docs feature on
top of the capabilities that CouchDB will make available.

I'm still coming to understand those new capabilities. I think I'll
need to see Damien's patch before I can have any considered opinion of
it. For instance, I am not comfortable holding a vote until we've had
time to understand the code.

On Thu, Feb 5, 2009 at 5:46 AM, Jan Lehnardt <jan@apache.org> wrote:
> Hi,
>
> *pouring water over the fire*
>
> The progression of this is very unfortunate. There was no formal discussion,
> neither on IRC or a mailing list. We are all aware of the ASF ways of
> running
> a project and we didn't handle that one well.
>
> Apologies.

I agree that we as a PMC should strive to be more transparent in the
future. Making the transactional _bulk_docs API available in the first
place was a hard decision, and it's not clear that it was the right
decision (although it did make testing ACID transactions easier).

The CouchDB project came into the Incubator with a lot of momentum and
direction, and I consider part of my role with the project, to help
insulate Damien from the mailing-list chatter, especially when he's
deep in code. I acknowledge that could be a mistake as well, if it
leads to community misapprehension.

> The whole thing started because I closed a bug with a comment that
> there must be an _upcoming discussion_.

I sympathize with Antony's predicament. He's been using bulk doc
transactions in a high-pressure environment, and it works for him.
It's understandable that he'd be upset, first hearing about the patch
like this.

I know that the long-standing vision of Couch doesn't include special
API exceptions for when you are running on a single node. And I'm a
little afraid that the transactional doc commits Antony wants us to
keep, are only a mirage, which would lead to trouble anyway, when
distributed systems are involved.

I think a consensus algorithm client library could provide the same
semantics as the current feature, even on a cluster. An implementation
would let Antony keep his feature, even on larger clusters. It could
easily be included as an Erlang plugin.

Couch has a way of forcing developers to rethink their applications in
order to make them fit into its mode of operation. I think if we
approach the problem from a technical angle, it will help everyone to
have an informed opinion about the patch.

I'd been hoping to hold this discussion until Damien makes his code
available, as I think that's when it'd be most appropriate.

Antony, maybe it would help for you to explain just exactly what you
wouldn't be able to do, without the bulk docs API. It will help to
inform people about the technical issue.

Sincerely,
Chris

-- 
Chris Anderson
http://jchris.mfdz.com