couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adam Kocoloski <kocol...@apache.org>
Subject Re: Preserving seq order through replication
Date Fri, 05 Mar 2010 17:10:49 GMT
On Mar 5, 2010, at 11:56 AM, J Chris Anderson wrote:

> On Mar 5, 2010, at 8:08 AM, Dirkjan Ochtman wrote:
> 
>> Hi there,
>> 
>> I recently started using continuous replication to propagate a
>> document stream from a producing server to a consuming server. The
>> producing server is fed by a single process that puts messages in, and
>> on the consuming server we have a process reading the continuous
>> changes feed to get messages back out and process them. We migrated to
>> this from having a single server that both processes talked to because
>> the connection between the servers isn't always reliable,
>> unfortunately.
>> 
>> This worked rather well for about 50 hours, but now it turns out that
>> in some cases the continuous replication doesn't always preserve the
>> ordering of the sequence. That is, the insertion order, which is
>> consistent on the producing server (as expected, as there's a single
>> feeding process doing a single update per document) doesn't match the
>> order in the continuous feed of changes on the consuming server.
>> 
>> This is kind of a WTF. Could it be fixed?
> 
> I don't think this is a bug. Preserving order across replicas is definitely something
that won't scale across a cluster (especially if it is partitioned), so we wouldn't want to
attempt guarantees like that at smaller scales.
> 
> Of course, if you have a field you'd like to order operations on, you can mark your documents
with it and use view queries to drive the asynchronous processing.
> 
> Chris

With that said, making replication preserve the update order is probably not very difficult
or expensive to do.  Best,

Adam
Mime
View raw message