couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Anderson <jch...@apache.org>
Subject Re: Bulk Docs
Date Fri, 13 Mar 2009 00:15:14 GMT
On Thu, Mar 12, 2009 at 3:09 PM, Tim Parkin <info@timparkin.co.uk> wrote:
> Damien Katz wrote:
>> Atomic bulk docs is in the patch, it just doesn't do conflict checking.
>> If any docs are conflicts, they are saved anyway as conflicts. This
>> means it's really for message queue functionality, not database
>> consistency, your data is safe and committed but might not be
>> immediately available or consistent between docs. The reasons we are
>> removing all or nothing with conflict checking as it doesn't work with
>> replication (both offline and clustering) as docs are not replicated in
>> a single transaction or even in update order. And getting it to work
>> with partitioning would cause unacceptable write performances. If we
>> leave it, people will rely on the behavior not understanding it doesn't
>> really work with the rest of CouchDB.
>
> I know it won't work when I replicate but I also know various other
> things need to be taken to account when I start to distribute my
> application too. I'd like to be able to choose when to consider these
> things.
>
>>
>> So if you are currently using bulk docs to guarantee inter-document
>> consistency, it already doesn't work with replication. It only works on
>> a single machine, so no master-slave and no hot stand-by setup would
>> work as neither are guaranteed to be in a consistent state at any point.
>>
>
> Yes but it does work on a single node and allows *better* local 'shard'
> consistency and also allows simpler conflict checking whilst an
> application is growing (I can spend my initial investment money on
> developing application code, not dealing with distribution before it's
> needed).
>

What I'm curious is why your application needs inter-document
consistency. It seems like most of these problems just go away if you
put all the data that needs to be edited transactionally into a single
document. Bulk doc consistency seems like an attempt to sweep under
the rug the fact that you have interdependencies. I know it's a
pragmatic difference, and that a bulk transaction isn't really a
document (but it basically becomes a document if you want the
transaction to stay together as it passes through replications).

Anyway, it may be worth rethinking multi-doc transactions as a
combination of single doc updates and views. Eg: the bank example
where you have a document per transaction, and an account is a view
that sums up the values of all the transactions a given account is
involved in...

Chris

-- 
Chris Anderson
http://jchris.mfdz.com

Mime
View raw message