couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Zachary Zolton <>
Subject Re: Strategy for reliable _changes feed workers
Date Mon, 05 Mar 2012 22:28:06 GMT
> There's an issue with this in that you have an eventually consistent
> database and you require a consistent update (read my write) on a cluster.
> Yes you'd need to be a bit unlucky to have this happen but the race
> condition isn't that impossible to meet in a big enough system.
> Thats especially true if "workers" are replicating from the central
> database, writing that update to the local DB and then replicating back,
> resulting in conflicts, confusion and horror. If your workers do something
> that isn't idempotent (e.g. launch a rocket) you kind of need a handshake
> between the different clients, it's not enough to just take an unprocessed
> document, you have to make sure that no one else has taken it, too. In the
> case of replicating workers you have a consistent winner and a set of
> conflicts. If your change to the database is the winner you do the work, if
> it's a conflict you don't.
> Cheers
> Simon

You're right about non-idempotent operations, I guess the workers should
try put the document into an "in-process" state without receiving an update
conflict from couch?

And then, you need another background watcher to look for workers that have
had a doc in-process longer some timeout value. So, that's like another
program and view... Maybe a proper work queue is the Right Tool?

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message