couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Damien Katz <>
Subject Re: couchdb transactions changes
Date Mon, 09 Feb 2009 05:36:18 GMT

On Feb 8, 2009, at 11:27 PM, Antony Blakey wrote:

> On 09/02/2009, at 2:35 PM, Paul Davis wrote:
>> There is no concept of an "MVCC boundary" anywhere in the code that
>> I'm aware of.
> Database updates create an MVCC commit, reads are all wrt an MVCC  
> commit. MVCC boundaries e.g. commit points, are a fundamental port  
> of the Couch low-level architecture. When _bulk_docs was ACID, they  
> were exposed in the user-level API.
>> I think the bigger point here is that what you're asking for violates
>> a huge swath of assumptions baked into the core of CouchDB. Asking
>> CouchDB to do consistent inter-document writes is going to require  
>> you
>> to either change a large amount of internal code or write some very
>> specific app code to get what you want.
> But it already did consistent inter-document writes - the removal of  
> that is what this discussion is about.
>> You may be able to get atomic
>> interdocument updates on a single node, but this is violated if you  
>> do
>> so much as try and replicate.
> And 'so much as try and replicate' is the issue, because the  
> replication model varies for different use cases. In my previous  
> posts you'll see that I'm promoting the idea that the local,  
> exclusive-replication use-case is significant, and useful. The are  
> useful models where replication is a fundamentally different  
> operation than local use.
>> IMO, it would be better to not support _bulk_docs for exactly this
>> reason. People that use _bulk_docs will end up assuming that the
>> atomic properties will carry over into places it doesn't actually get
>> passed on to.
> But it can for local operations, and replications conflicts can be  
> dealt with separately from normal operation.
>> It occurs to me that once you get to the point of writing source and
>> target database locking, you no longer need _bulk_docs. You'd have
>> enough code to do all the atomic interdoc writes you need.
> Only by giving up all local concurrency. Locking is only wrt.  
> replication vs. local operation. And I think the most recent emails  
> are showing that source locking is not as black-and-white as you  
> think - it's only wrt compaction, and even then  I think it's  
> restricted to a requirement to no compact past the MVCC state being  
> used by the replication process, which IMO is a trivial issue  
> because compaction cannot invalidate the head MVCC state, and  
> replication request will always use the head state in effect at  
> request-time.
>> Though it'd
>> be rather un-couchy.
> CouchDB has wide applicability, and what you regard as un-couchy is  
> only relative to a certain use-case. I'm trying to promote a more  
> generous interpretation of what CouchDB is, and can be.

I see the critical problem being with consistent updates of  
replication. Unless you do it one big transaction, the intermediate  
replication states of the database are inconsistent, so the target  
database is unusable during replication. A bulk transaction is limited  
in how many docs it can handle, so it only works for smallish  
databases. That alone means MVCC replication isn't useful in the  
general case.

But for your purposes, it's maybe possible. You'll need to write a  
special replicator and create a single HTTP request to give you  
everything from the source in one go. Then you'll need to write-lock  
(or disable) the target database during replication, unless they are  
always small databases in which case the special replicator can use a  
single bulk transaction.

You'll also need to serialize the updates to the database in the  
application layer and add the conflict checking there. That will give  
you the desired transaction semantics.

If you build what I've described, and assuming you can live with the  
limitations, you will have a always consistent one-way replication  
document distribution platform.


> Antony Blakey
> --------------------------
> CTO, Linkuistics Pty Ltd
> Ph: 0438 840 787
> Human beings, who are almost unique in having the ability to learn  
> from the experience of others, are also remarkable for their  
> apparent disinclination to do so.
>  -- Douglas Adams

View raw message