incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Damien Katz <dam...@apache.org>
Subject Re: replication usage? creating dupes?
Date Tue, 15 Jul 2008 23:50:48 GMT
Looks like the cause of the duplicates is a problem in the HTTP layer.  
It seems POST requests can fail to respond despite success, and then  
are retried by the HTTP client. I'm not sure if this is an HTTP client  
or a TCP layer thing that's doing the re-transmission.

There seems to be a bug in Mochiweb that causes this to happen  
frequently when used by Safari, where it suddenly breaks the  
connection to the client. I'm not sure why it does this.

However, upon investing this it appears this is always a potential  
problem, POSTing documents isn't idempotent, so a success followed by  
a network failure and retry can cause dupes. Therefore, don't use POST  
to create documents, generate the document ID yourself and use PUT.  
This will prevent duplicate documents from ever happening due to  
network failure.

But it should not be the case that replication causes duplicates.  
Replication uses only idempotent requests, making that unlikely (but I  
won't say impossible).

-Damien

On Jul 15, 2008, at 3:24 PM, Chris Anderson wrote:

> On Tue, Jul 15, 2008 at 10:40 AM, John Evans <john@jpevans.com> wrote:
>> FWIW I've been running into these exact same issues.
>
> I'll chime in with a me too. I've worked around it by writing a
> client-side replicator, that loads up all docs from both dbs, and
> copies any missing docs over to the target db. My requirements mean I
> don't need to worry about multiple versions of a doc, so this has
> worked for me for the time being. It would be more convenient to have
> fast, reliable replication.
>
> I'll commit the replicator to CouchRest this week.
>
> -- 
> Chris Anderson
> http://jchris.mfdz.com


Mime
View raw message