couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Filipe Manana (JIRA)" <j...@apache.org>
Subject [jira] Updated: (COUCHDB-864) multipart/related PUT's always close the connection.
Date Sun, 22 Aug 2010 17:38:17 GMT

     [ https://issues.apache.org/jira/browse/COUCHDB-864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Filipe Manana updated COUCHDB-864:
----------------------------------

    Attachment: mp_doc_put_http_pipeline.patch

So, a tested patch that assure the PUT multipart/related doc API doesn't read more data than
it needs (to avoid consuming data from a subsequent request in an HTTP pipeline) and to allow
HTTP request with chunked transfer-encoding as well.
This HTTP pipeline issue is important for the replicator, since it uses this API very frequently.

Etap test included.

If no one has anything against this or the patch itself, I'll commit it in a few days.

> multipart/related PUT's always close the connection.
> ----------------------------------------------------
>
>                 Key: COUCHDB-864
>                 URL: https://issues.apache.org/jira/browse/COUCHDB-864
>             Project: CouchDB
>          Issue Type: Bug
>          Components: Database Core
>            Reporter: Robert Newson
>         Attachments: chunked.erl, mp_doc_put_http_pipeline.patch, mp_pipeline.patch
>
>
> I noticed that mochiweb always closes the connection when doing a multipart/related PUT
(to insert the JSON document and accompanying attachments in one call). Ultimately it's because
we call recv(0) and not recv_body, thus consuming more data than we actually process. Mochiweb
notices that there is unread data on the socket and closes the connection.
> This impacts replication with attachments, as I believe they go through this code path
(and, thus, are forever reconnecting).
> The code below demonstrates a fix for this issue but isn't good enough for trunk. Adam
provided the important process dictionary fix.
> ---
>  src/couchdb/couch_doc.erl      |    1 +
>  src/couchdb/couch_httpd_db.erl |   13 +++++++++----
>  2 files changed, 10 insertions(+), 4 deletions(-)
> diff --git a/src/couchdb/couch_doc.erl b/src/couchdb/couch_doc.erl
> index 5009f8f..f8c874b 100644
> --- a/src/couchdb/couch_doc.erl
> +++ b/src/couchdb/couch_doc.erl
> @@ -455,6 +455,7 @@ doc_from_multi_part_stream(ContentType, DataFun) ->
>      Parser ! {get_doc_bytes, self()},
>      receive 
>      {doc_bytes, DocBytes} ->
> +        erlang:put(mochiweb_request_recv, true),
>          Doc = from_json_obj(?JSON_DECODE(DocBytes)),
>          % go through the attachments looking for 'follows' in the data,
>          % replace with function that reads the data from MIME stream.
> diff --git a/src/couchdb/couch_httpd_db.erl b/src/couchdb/couch_httpd_db.erl
> index b0fbe8d..eff7d67 100644
> --- a/src/couchdb/couch_httpd_db.erl
> +++ b/src/couchdb/couch_httpd_db.erl
> @@ -651,12 +651,13 @@ db_doc_req(#httpd{method='PUT'}=Req, Db, DocId) ->
>      } = parse_doc_query(Req),
>      couch_doc:validate_docid(DocId),
>      
> +    Len = couch_httpd:header_value(Req,"Content-Length"),
>      Loc = absolute_uri(Req, "/" ++ ?b2l(Db#db.name) ++ "/" ++ ?b2l(DocId)),
>      RespHeaders = [{"Location", Loc}],
>      case couch_util:to_list(couch_httpd:header_value(Req, "Content-Type")) of
>      ("multipart/related;" ++ _) = ContentType ->
>          {ok, Doc0} = couch_doc:doc_from_multi_part_stream(ContentType,
> -                fun() -> receive_request_data(Req) end),
> +                fun() -> receive_request_data(Req, Len) end),
>          Doc = couch_doc_from_req(Req, DocId, Doc0),
>          update_doc(Req, Db, DocId, Doc, RespHeaders, UpdateType);
>      _Else ->
> @@ -775,9 +776,13 @@ send_docs_multipart(Req, Results, Options) ->
>      couch_httpd:send_chunk(Resp, <<"--">>),
>      couch_httpd:last_chunk(Resp).
>  
> -receive_request_data(Req) ->
> -    {couch_httpd:recv(Req, 0), fun() -> receive_request_data(Req) end}.
> -    
> +receive_request_data(Req, undefined) ->
> +    receive_request_data(Req, 0);
> +receive_request_data(Req, Len) when is_list(Len)->
> +    Remaining = list_to_integer(Len),
> +    Bin = couch_httpd:recv(Req, Remaining),
> +    {Bin, fun() -> receive_request_data(Req, Remaining - iolist_size(Bin)) end}.
> +
>  update_doc_result_to_json({{Id, Rev}, Error}) ->
>          {_Code, Err, Msg} = couch_httpd:error_info(Error),
>          {[{id, Id}, {rev, couch_doc:rev_to_str(Rev)},
> -- 
> 1.7.2.2
> Umbra

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message