couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Damien Katz (JIRA)" <j...@apache.org>
Subject [jira] Commented: (COUCHDB-583) adding ?compression=(gzip|deflate) optional parameter to the attachment download API
Date Mon, 30 Nov 2009 16:36:20 GMT

    [ https://issues.apache.org/jira/browse/COUCHDB-583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12783732#action_12783732
] 

Damien Katz commented on COUCHDB-583:
-------------------------------------

One problem I think I see with the patch is that we are compressing regardless of mime type.
For already compressed files (image, music and video), it does nothing but add CPU overhead.

Perhaps we need a separate user editable ini file to specify compressable or non-compressable
files (would probably be too big for the regular ini file). What do other web servers do?

Also, a potential optimization is to compress the file while writing to disk, and serve the
compressed bytes directly to clients that can handle it, and decompressed for those that can't.
For compressable types, it's a win for both disk IO for reads and writes, and CPU on read.

> adding ?compression=(gzip|deflate) optional parameter to the attachment download API
> ------------------------------------------------------------------------------------
>
>                 Key: COUCHDB-583
>                 URL: https://issues.apache.org/jira/browse/COUCHDB-583
>             Project: CouchDB
>          Issue Type: New Feature
>          Components: HTTP Interface
>         Environment: CouchDB trunk revision 885240
>            Reporter: Filipe Manana
>         Attachments: jira-couchdb-583-1st-try-trunk.patch, jira-couchdb-583-2nd-try-trunk.patch
>
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> The following new feature is added in the patch following this ticket creation.
> A new optional http query parameter "compression" is added to the attachments API.
> This parameter can have one of the values:  "gzip" or "deflate".
> When asking for an attachment (GET http request), if the query parameter "compression"
is found, CouchDB will send the attachment compressed to the client (and sets the header Content-Encoding
with gzip or deflate).
> Further, it adds a new config option "treshold_for_chunking_comp_responses" (httpd section)
that specifies an attachment length threshold. If an attachment has a length >= than this
threshold, the http response will be chunked (besides compressed).
> Note that using non chunked compressed  body responses requires storing all the compressed
blocks in memory and then sending each one to the client. This is a necessary "evil", as we
only know the length of the compressed body after compressing all the body, and we need to
set the "Content-Length" header for non chunked responses. By sending chunked responses, we
can send each compressed block immediately, without accumulating all of them in memory.
> Examples:
> $ curl http://localhost:5984/testdb/testdoc1/readme.txt?compression=gzip
> $ curl http://localhost:5984/testdb/testdoc1/readme.txt?compression=deflate
> $ curl http://localhost:5984/testdb/testdoc1/readme.txt   # attachment will not be compressed
> $ curl http://localhost:5984/testdb/testdoc1/readme.txt?compression=rar   # will give
a 500 error code
> Etap test case included.
> Feedback would be very welcome.
> cheers

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message