couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Filipe David Manana <fdman...@gmail.com>
Subject Re: [jira] Commented: (COUCHDB-583) storing attachments in compressed form and serving them in compressed form if accepted by the client
Date Tue, 26 Jan 2010 11:01:22 GMT
Hi Robert,

That's interesting.
I think that abstraction is doable, but maybe not trivial.

In your idea, you plan to always decrypt the docs/attachments before sending
them to the client?


On Tue, Jan 26, 2010 at 11:34 AM, Robert Newson <robert.newson@gmail.com>wrote:

> fyi: I have a (much delayed) plan to work up an encryption patch for
> documents and attachments. Since encryption and compression both apply
> at the same level (and the order of the two is important), I wonder if
> that would change the approach taken here? That is, would an
> abstraction that allowed a chain of transformations when storing (and
> the reverse chain when retrieving) be in order?
>
> B.
>
> On Tue, Jan 26, 2010 at 8:02 AM, Filipe Manana (JIRA) <jira@apache.org>
> wrote:
> >
> >    [
> https://issues.apache.org/jira/browse/COUCHDB-583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12804946#action_12804946]
> >
> > Filipe Manana commented on COUCHDB-583:
> > ---------------------------------------
> >
> > @Chris
> >
> > Good point, I totally agree.
> > It would be interesting to test with real couchapps, real data and see
> how worth it really is.
> >
> > A 10Mb text file, for instance, was compressed to about 100Kb in one of
> my tests.
> >
> > Also, as for the minified JavaScript files for example, it's still worth
> compressing them. For example, the minified Ext JS lib file (
> http://www.extjs.com,  ext-all.js) is about 630Kb big. Compressed with
> gzip stays at about 170Kb, therefore a reasonably good size reduction.
> >
> > As Damien said in a previous comment, not only saves disk space but also
> reduces disk IO (attachment download requests, compaction).
> >
> > I also look forward to see the impact on real, production level,
> applications.
> >
> >> storing attachments in compressed form and serving them in compressed
> form if accepted by the client
> >>
> ----------------------------------------------------------------------------------------------------
> >>
> >>                 Key: COUCHDB-583
> >>                 URL: https://issues.apache.org/jira/browse/COUCHDB-583
> >>             Project: CouchDB
> >>          Issue Type: New Feature
> >>          Components: Database Core, HTTP Interface
> >>         Environment: CouchDB trunk
> >>            Reporter: Filipe Manana
> >>         Attachments: couchdb-583-trunk-10th-try.patch,
> couchdb-583-trunk-11th-try.patch, couchdb-583-trunk-12th-try.patch,
> couchdb-583-trunk-13th-try.patch, couchdb-583-trunk-14th-try-git.patch,
> couchdb-583-trunk-15th-try-git.patch, couchdb-583-trunk-3rd-try.patch,
> couchdb-583-trunk-4th-try-trunk.patch, couchdb-583-trunk-5th-try.patch,
> couchdb-583-trunk-6th-try.patch, couchdb-583-trunk-7th-try.patch,
> couchdb-583-trunk-8th-try.patch, couchdb-583-trunk-9th-try.patch,
> jira-couchdb-583-1st-try-trunk.patch, jira-couchdb-583-2nd-try-trunk.patch
> >>
> >>
> >> This feature allows Couch to gzip compress attachments as they are being
> received and store them in compressed form.
> >> When a client asks for downloading an attachment (e.g. GET
> somedb/somedoc/attachment.txt), the attachment is sent in compressed form if
> the client's http request has gzip specified as a valid transfer encoding
> for the response (using the http header "Accept-Encoding"). Otherwise couch
> decompresses the attachment before sending it back to the client.
> >> Attachments are compressed only if their MIME type matches one of those
> listed in a separate config file. Compression level is also configurable in
> the default.ini file.
> >> This follows Damien's suggestion from 30 November:
> >> "Perhaps we need a separate user editable ini file to specify
> compressable or non-compressable files (would probably be too big for the
> regular ini file). What do other web servers do?
> >> Also, a potential optimization is to compress the file while writing to
> disk, and serve the compressed bytes directly to clients that can handle it,
> and decompressed for those that can't. For compressable types, it's a win
> for both disk IO for reads and writes, and CPU on read."
> >> Patch attached.
> >
> > --
> > This message is automatically generated by JIRA.
> > -
> > You can reply to this email to add a comment to the issue online.
> >
> >
>



-- 
Filipe David Manana,
fdmanana@gmail.com
PGP key - http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xC569452B

"Reasonable men adapt themselves to the world.
Unreasonable men adapt the world to themselves.
That's why all progress depends on unreasonable men."

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message