couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Antony Blakey <>
Subject Re: couchdb transactions changes
Date Mon, 09 Feb 2009 03:29:24 GMT

On 09/02/2009, at 1:07 PM, Damien Katz wrote:

> On Feb 8, 2009, at 9:24 PM, Chris Anderson wrote:
>> On Sun, Feb 8, 2009 at 5:54 PM, Damien Katz <>  
>> wrote:
>>> It's possible to use MVCC for replication. You'll need to create  
>>> special
>>> HTTP command to return you all the documents you are interested in  
>>> a single
>>> request, and a special replicator that uses that command and loads  
>>> those
>>> documents and writes them to the destination.

>> This sound a lot like the notification view Damien's been talking
>> about, where clients can register to be told about database updates
>> that match particular functions.
>> The main problem I see with MVCC replication is that if it dies in  
>> the
>> middle, you might not be able to restart it right where you left off.
> That would be a big problem of replicating huge databases.  
> Everything must come over in one transaction.

You could still do that incrementally e.g. it wouldn't have to load in  
a single request. The key is that the replication shows MVCC  
boundaries i.e. add a marker in the replication stream to indicate  
when you passed an MVCC commit point. The current model would ignore  
such markers.- nothing else is required I think. You could even cycle  
as long as there were new MVCC states, which would give the same  
'includes-updates-as-they-come-in' form of replication, but with  
somewhat more consistency. If these restart points were included in  
the replication stream, then systems that wanted to allow replication  
rollback (see below) could reset the rollback MVCC state when they get  
an end-of-MVCC state marker.

If you cared however, and if your application model allowed it, then  
you could guarantee consistency by only accepting completed  
replications. I imagine you would need to be able to 'undo' an  
incomplete replication, which would be a matter of allowing the db to  
be rolled back to the MVCC state that was in effect when the  
replication started. This would prevent permanent lockup, and I'm sure  
you'd want this facility to be enabled/disabled in configuration.

I want to stress that I know this is only useful for a certain class  
of use, but I don't think it negatively impacts other uses, so only  
those uses that want it would pay for it.

Also, this doesn't resolve the cluster-ACID issue, but I'm confident  
there is a solution there that doesn't impact the clustered/non- 
exclusive-replication/constant-conflict model.

> If you want consistency on the target, you'll have to write lock the  
> database (not a concept couchdb really has)

I think that's an application-level concept, but I can imagine a patch  
that allows it in CouchDB. Still, I'd do that at an application level,  
because I'm in user-triggered exclusive replication mode.

> until the whole replication completes, or write all the docs in one  
> big bulk transaction, which won't work for large databases. And  
> while that transaction is occurring, neither the source database or  
> target database file itself be compacted (the compaction will take  
> place, but the old file can't be freed until the full transaction  
> completes).

Yes, but once again I think there are valid use-cases that allow that  
e.g. mine at least.

Antony Blakey
CTO, Linkuistics Pty Ltd
Ph: 0438 840 787

The trouble with the world is that the stupid are cocksure and the  
intelligent are full of doubt.
   -- Bertrand Russell

View raw message