incubator-couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christopher Lenz <cml...@gmx.de>
Subject Fwd: Need help debugging mochiweb/Safari HTTP problems
Date Wed, 23 Jul 2008 21:23:19 GMT
Hey all,

since we've moved CouchDB to MochiWeb a couple months ago we've been  
getting intermittent errors that seem to be at the TCP or low-level  
HTTP level. We've not had much luck so far figuring out the cause the  
problem, but maybe someone here has an idea; please see Damien's  
description below...

Begin forwarded message:
> From: Damien Katz <damien@apache.org>
> Date: 23. Juli 2008 21:01:28 MESZ
> To: couchdb-dev@incubator.apache.org
> Subject: Need help debugging mochiweb/Safari HTTP problems
> Reply-To: couchdb-dev@incubator.apache.org
>
> Right now we are having a major problem with HTTP request being  
> retried. This problem is responsible for the test suite failures  
> seen constantly in Safari (though others report similar failures in  
> Firefox, I've not seen them myself). And not just test suite  
> failures, some are seeing the same behavior in production.
>
> The major symptoms of this problem:
> 1. Mysterious conflict - You get a conflict error saving a document  
> to the db. When you examine the existing db document, it's already  
> got your changes.
> 2. Duplicate document - When creating a new document via POST, you  
> occasionally get 2 new documents created instead of one.
>
> #1 is annoying but not too serious, no data is lost or corrupted. #2  
> is a bit more dangerous, because you could consider the database  
> corrupted by having the duplicate document. (depends on what  
> problems it would cause for your app)
>
> What is happening in both these cases is the HTTP requests are  
> getting sent and processed twice. The first request is given to  
> CouchDB and is handled, but when CouchDB attempts to send the  
> response, the connection is reset (apparently). Then another  
> identical HTTP request comes in and the request is processed again.
>
> I am not a TCP expert. but by viewing the network requests via  
> tcpdump, it is obvious the request packets, 1 header and 1 body  
> packet, are getting resent from the client to the server. I do not  
> know if the packets are being resent at the TCP level, or if the  
> HTTP client in safari is retrying the request after getting a TCP  
> error.
>
> I do not know why the network error or subsequent resend is  
> happening. I can only confirm that it *is* happening. If this is at  
> the TCP level, then it means we definitely need to do away with the  
> non-idempotent POST to create new documents.
>
> I think we do anyway though. While this network error should not be  
> happening, it did expose an interesting problem with our use of POST  
> for document creation. The problem is the generated id for the  
> document is a UUID generated server side, so the server has no way  
> to distinguish if a request is a new request or a resend of an  
> already processed request, and so generates another UUID and thus  
> creates another new document. But if the UUID is generated by the  
> client, then the resend will cause a conflict error, that UUID  
> already exists in the DB, thus eliminating the duplicate data.
>
> However, we still need to figure out why this is happening in the  
> first place. Why is the connection being reset and why is the  
> request being retried?
>
> If anyone want to try to debug this, here is what I've been doing:
> 1. Run a packet sniffer for local port 5984 and start couchdb
> 2. Got to http://127.0.0.1/_utils/,  click the "Test Suite" link
> 3. Run the "basics" test manually until you see a "conflict error"  
> exception in test result. (This exception stops the test executing.  
> I don't try to debug other test failures, since the test keeps on  
> running after the failure)
> 4. The last few requests will be the duplicated requests. There is  
> information about the packets, but I don't know how to interpret it.
>
> Any help and input appreciated.
>
> -Damien


Thanks,
--
Christopher Lenz
   cmlenz at gmx.de
   http://www.cmlenz.net/



Mime
View raw message