couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Filipe Manana (JIRA)" <>
Subject [jira] Commented: (COUCHDB-583) storing attachments in compressed form and serving them in compressed form if accepted by the client
Date Tue, 22 Dec 2009 13:26:29 GMT


Filipe Manana commented on COUCHDB-583:


you're comments refer to the first 3 patches' implementation. The 4th and latest follow Damien's
idea (comment from the 30th November).

Check the last patch:  	couchdb-583-trunk-6th-try.patch

The approach is completely different. There's no use of the query parameter ?compression=(gzip|deflate)
and no longer that block buffering thing for compression / decompression :) With the latest
ones the attachment are compressed and stored in compressed form (if their mime type matches
one of those in the config file).

As soon as a data chunk is received from the client, it is compressed with a zlib stream and
written to disk. Decompression follows the same idea - 1 block is read from the disk, compressed
and a chunk sent to the client. No need to buffer things. I figured out how to use zlib for
incremental gzip compression/decompression.

The "reshold_for_chunking_comp_responses" is completely gone also. HTTP content-encoding is
now negotiated.

After analying the patch, let me know if the implementation is ok and how to simplify it further.


> storing attachments in compressed form and serving them in compressed form if accepted
by the client
> ----------------------------------------------------------------------------------------------------
>                 Key: COUCHDB-583
>                 URL:
>             Project: CouchDB
>          Issue Type: New Feature
>          Components: Database Core, HTTP Interface
>         Environment: CouchDB trunk
>            Reporter: Filipe Manana
>         Attachments: couchdb-583-trunk-3rd-try.patch, couchdb-583-trunk-4th-try-trunk.patch,
couchdb-583-trunk-5th-try.patch, couchdb-583-trunk-6th-try.patch, jira-couchdb-583-1st-try-trunk.patch,
> This feature allows Couch to gzip compress attachments as they are being received and
store them in compressed form.
> When a client asks for downloading an attachment (e.g. GET somedb/somedoc/attachment.txt),
the attachment is sent in compressed form if the client's http request has gzip specified
as a valid transfer encoding for the response (using the http header "Accept-Encoding"). Otherwise
couch decompresses the attachment before sending it back to the client.
> Attachments are compressed only if their MIME type matches one of those listed in a separate
config file. Compression level is also configurable in the default.ini file.
> This follows Damien's suggestion from 30 November:
> "Perhaps we need a separate user editable ini file to specify compressable or non-compressable
files (would probably be too big for the regular ini file). What do other web servers do?
> Also, a potential optimization is to compress the file while writing to disk, and serve
the compressed bytes directly to clients that can handle it, and decompressed for those that
can't. For compressable types, it's a win for both disk IO for reads and writes, and CPU on
> Patch attached.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message