couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Filipe Manana (JIRA)" <>
Subject [jira] Updated: (COUCHDB-583) adding ?compression=(gzip|deflate) optional parameter to the attachment download API
Date Tue, 08 Dec 2009 20:35:18 GMT


Filipe Manana updated COUCHDB-583:

    Attachment: couchdb-583-trunk-3rd-try.patch


Here follows the patch implementing Damien's suggestion.

A new key under the httpd section is added to the config file. Its value is the name of a
file that lists which MIME types are worth compressing (text based types for e.g.).

When an attachment is uploaded (using the standalone attachment API), if its type matches
one of the MIME types listed in the former file, the attachment is stored with gzip compression.

When a client requests the download of an attachment:

1) if the HTTP request message has an Accept-Encoding header listing gzip as an acceptable
response body encoding method, the attachment is sent unmodified to the client (in gzip form
and in a chunked HTTP message)

2) if the HTTP request doesn't include that header, or if it includes but it mentions gzip
as not acceptable, the attachment is uncompressed (on the fly) and sent to the client (in
a chunked HTTP response)

An Etap test, covering 7 scenarios, is included.
Note that it breaks some other tests (for example, tests that upload a text attachment and
then downloaded it to check its content). I'll fix those tests after your feedback. This is
a preliminary version.

Feedback (specially from the commiters) regarding the implementation is very welcome.

Filipe Manana (fdmanana)

> adding ?compression=(gzip|deflate) optional parameter to the attachment download API
> ------------------------------------------------------------------------------------
>                 Key: COUCHDB-583
>                 URL:
>             Project: CouchDB
>          Issue Type: New Feature
>          Components: HTTP Interface
>         Environment: CouchDB trunk revision 885240
>            Reporter: Filipe Manana
>         Attachments: couchdb-583-trunk-3rd-try.patch, jira-couchdb-583-1st-try-trunk.patch,
>   Original Estimate: 24h
>  Remaining Estimate: 24h
> The following new feature is added in the patch following this ticket creation.
> A new optional http query parameter "compression" is added to the attachments API.
> This parameter can have one of the values:  "gzip" or "deflate".
> When asking for an attachment (GET http request), if the query parameter "compression"
is found, CouchDB will send the attachment compressed to the client (and sets the header Content-Encoding
with gzip or deflate).
> Further, it adds a new config option "treshold_for_chunking_comp_responses" (httpd section)
that specifies an attachment length threshold. If an attachment has a length >= than this
threshold, the http response will be chunked (besides compressed).
> Note that using non chunked compressed  body responses requires storing all the compressed
blocks in memory and then sending each one to the client. This is a necessary "evil", as we
only know the length of the compressed body after compressing all the body, and we need to
set the "Content-Length" header for non chunked responses. By sending chunked responses, we
can send each compressed block immediately, without accumulating all of them in memory.
> Examples:
> $ curl http://localhost:5984/testdb/testdoc1/readme.txt?compression=gzip
> $ curl http://localhost:5984/testdb/testdoc1/readme.txt?compression=deflate
> $ curl http://localhost:5984/testdb/testdoc1/readme.txt   # attachment will not be compressed
> $ curl http://localhost:5984/testdb/testdoc1/readme.txt?compression=rar   # will give
a 500 error code
> Etap test case included.
> Feedback would be very welcome.
> cheers

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message