Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@couchdb.apache.org
Received-SPF: pass (athena.apache.org: domain of robert.newson@gmail.com
 designates 209.85.160.52 as permitted sender)
References: <4FC0BFA5.6030004@cedarsoft.com>
From: Robert Newson <robert.newson@gmail.com>
In-Reply-To: <4FC0BFA5.6030004@cedarsoft.com>
Mime-Version: 1.0 (1.0)
Date: Sat, 26 May 2012 14:21:32 +0100
Message-ID: <260788549962614027@unknownmsgid>
Subject: Re: Misterious 409 conflicts: Duplicated PUT requests
To: "user@couchdb.apache.org" <user@couchdb.apache.org>
Content-Type: multipart/alternative; boundary=047d7b339c0f252f1904c0f05dad

--047d7b339c0f252f1904c0f05dad
Content-Type: text/plain; charset=ISO-8859-1

Jersey can use either the JDK's http client or Apache HttpClient,
which are you using?

I know that the Apache HttpClient will resubmit requests under some
conditions (and you can disable it).

http://hc.apache.org/httpcomponents-client-ga/tutorial/html/fundamentals.html#d5e281

The default retry handler assumes PUT is idempotent but this is not
true for CouchDB. For it to be true, we would have to send an
identical response to the 2nd attempt as we did to the first, which,
as you can see, we don't.

CouchDB is behaving correctly, your client is the problem.

Sent from my iPhone

On 26 May 2012, at 12:34, Johannes Schneider <mailings@cedarsoft.com> wrote:

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi guys,


I have a big problem here. Very often my unit tests fail. I have asked
about this problem some time ago - but couldn't find that thread
anymore...

I have upgrade couchdb to 1.2.0 and run Wireshark to locate the problem.


Ok. I am uploading my design documents before each unit tests.
Sometimes (about 1 in 100) I get an 409 from couchdb.

The problem is, that there are two PUTs instead of just one. Of course
the second one fails with a 409.

But my client is *not* aware that it is sending the request twice.


Here is the relevant part of the couchdb.log:


[Sat, 26 May 2012 09:52:59 GMT] [info] [<0.23835.0>] 127.0.0.1 - - GET
/collustra_test$attachments/_design/rating 404
[Sat, 26 May 2012 09:52:59 GMT] [debug] [<0.23835.0>] httpd 404 error
response:
{"error":"not_found","reason":"missing"}

[Sat, 26 May 2012 09:52:59 GMT] [debug] [<0.23813.0>] 'PUT'
/collustra_test$attachments/_design/rating {1,1} from "127.0.0.1"
Headers: [{'Accept',"text/html, image/gif, image/jpeg, *; q=.2, */*;
q=.2"},
         {'Connection',"keep-alive"},
         {'Content-Length',"4910"},
         {'Content-Type',"text/plain"},
         {'Host',"localhost:5984"},
         {"Millis","1338025979060"},
         {'User-Agent',"Java/1.6.0_26"}]
[Sat, 26 May 2012 09:52:59 GMT] [debug] [<0.23813.0>] OAuth Params: []
[Sat, 26 May 2012 09:52:59 GMT] [debug] [<0.23836.0>] 'PUT'
/collustra_test$attachments/_design/rating {1,1} from "127.0.0.1"
Headers: [{'Accept',"text/html, image/gif, image/jpeg, *; q=.2, */*;
q=.2"},
         {'Connection',"keep-alive"},
         {'Content-Length',"4910"},
         {'Content-Type',"text/plain"},
         {'Host',"localhost:5984"},
         {"Millis","1338025979060"},
         {'User-Agent',"Java/1.6.0_26"}]
[Sat, 26 May 2012 09:52:59 GMT] [debug] [<0.23836.0>] OAuth Params: []
[Sat, 26 May 2012 09:52:59 GMT] [info] [<0.23813.0>] 127.0.0.1 - - PUT
/collustra_test$attachments/_design/rating 201
[Sat, 26 May 2012 09:52:59 GMT] [debug] [<0.23836.0>] Minor error in
HTTP request: conflict
[Sat, 26 May 2012 09:52:59 GMT] [debug] [<0.23836.0>] Stacktrace:
[{couch_db,update_doc,4,
                                   [{file,

"/tmp/a/build-couchdb/dependencies/couchdb/src/couchdb/couch_db.erl"},
                                    {line,429}]},
                                  {couch_httpd_db,update_doc,6,
                                   [{file,

"/tmp/a/build-couchdb/dependencies/couchdb/src/couchdb/couch_httpd_db.erl"},
                                    {line,931}]},
                                  {couch_httpd_db,do_db_req,2,
                                   [{file,

"/tmp/a/build-couchdb/dependencies/couchdb/src/couchdb/couch_httpd_db.erl"},
                                    {line,230}]},
                                  {couch_httpd,handle_request_int,5,
                                   [{file,

"/tmp/a/build-couchdb/dependencies/couchdb/src/couchdb/couch_httpd.erl"},
                                    {line,317}]},
                                  {mochiweb_http,headers,5,
                                   [{file,

"/tmp/a/build-couchdb/dependencies/couchdb/src/mochiweb/mochiweb_http.erl"},
                                    {line,136}]},
                                  {proc_lib,init_p_do_apply,3,
                                   [{file,"proc_lib.erl"},{line,227}]}]
[Sat, 26 May 2012 09:52:59 GMT] [info] [<0.23836.0>] 127.0.0.1 - - PUT
/collustra_test$attachments/_design/rating 409
[Sat, 26 May 2012 09:52:59 GMT] [debug] [<0.23836.0>] httpd 409 error
response:
{"error":"conflict","reason":"Document update conflict."}

[Sat, 26 May 2012 09:52:59 GMT] [debug] [<0.23837.0>] 'GET'
/collustra_test$attachments {1,1} from "127.0.0.1"
Headers: [{'Accept',"text/html, image/gif, image/jpeg, *; q=.2, */*;
q=.2"},
         {'Connection',"keep-alive"},
         {'Host',"localhost:5984"},
         {'User-Agent',"Java/1.6.0_26"}]


Now that seems to be very obvious...
But: On the client side I see just *1* PUT. I am using Jersey Client
API (Java).

Wiresharks shows that there are - in fact - two PUT.
But there is *no* response on the first PUT. Instead there are several
TCP errors (RST, ACK).

http://pastebin.com/17jV8MnA


So I have:

Client (Jersey): PUT
Wireshark: PUT, Errors but no response, PUT, response
CouchdB: PUT, (No Response?), PUT, Response


So I think the problem is, that the response to the first PUT is lost
somewhere - but executed successfully. And my client resends the
request and gets the 409...

So my question is basically: Does anybody have an idea how/why the
response is lost?


Thanks,

Johannes

- --
Johannes Schneider - blog.cedarsoft.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iQEcBAEBAgAGBQJPwL+iAAoJEAytD9R7Qv6d7c0IAMesCF/DZC91QA/uqm1v3rec
a8+npFGbvi0IhvMngJUKJtdCcAixn4FNQOoHeqY8HZf0BW5aHLtryk2/YzVzT4kg
5WG3FqdSqCCen92xfq0or+L5kRiC4D5U/jN1ZwFH5+LpyXyW6OQJ0N8kref8oDCh
WTJsvJVj1QKM5XQo6HcnCDKdQVRHgUNhgsoMPGp/ZJvcr7Ezh2j0vQkO99OQNyVb
VlFBKrwaFw+TV+x4jxabUPNW9GT5/g60zg8gwGWrO6bf3zCceUQA6Lk6zXxB5M/T
DBNieowdjAPJOpHn8xgvvJ3Z2wUavqk1KJhetQze+7iTEPxvDwgUW/yhXDLh8cQ=
=eUe/
-----END PGP SIGNATURE-----

--047d7b339c0f252f1904c0f05dad--