tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Gamache <cgama...@gmail.com>
Subject Re: sendFiles vs. compression
Date Wed, 19 Apr 2017 15:25:49 GMT
On Wed, Apr 19, 2017 at 9:26 AM, Christopher Schultz <
chris@christopherschultz.net> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> Chris,
>
> On 4/18/17 10:58 AM, Chris Gamache wrote:
> > Is there a way to create a split point where sendFile will handle
> > files of certain mime types (or all mime-types except for an
> > exclusion list of mime types) and/or of certain sizes while
> > compression will handle files of other mime-types and/or certain
> > sizes?
>
> You could configure a second DefaultServlet that matches specific
> patterns. You could also configure a "named" DefaultServlet and then
> use a Filter to decide if you want your CompressionDefaultServlet to
> handle your output versus your (likely "default") SendfileDefaultServlet
> .
>
Excellent suggestion. It's similar to the rewrite filter suggestion, but it
keeps web.xml simple which is one of my goals.


>
> > Both settings have a minimum file size that engages their mechanism
> > but to set up a division of labor I would think we would need all
> > of include/exclude and max/min for both sendfile() and compression.
> > Again, I could be missing something obvious by staring at the
> > problem too long.
>
> You could also pre-compress your static files.
>

I like that idea. It is a good way to take care of it in a targeted and
sane way.


>
> > @André and the rest of the listserv, In your opinions-- thinking
> > about the web site consumer's experience, and having to choose
> > either send sendfile() or compression-- is the juice worth the
> > squeeze so-to-speak keeping sendfile() and sending uncompressed
> > files, or is the better choice to enable compression at the expense
> > of direct static file access and save bandwidth?
>
> It really depends upon your use-case. If you have a lot of free CPU
> cycles, then using them instead of sendFile="true" will improve your
> overall throughput to the client.


> If you have a very CPU-intensive application (which most are NOT...
> most apps are just waiting around for a disk, database, etc. to
> respond) then maybe you want to be as CPU-efficient as possible and
> use sendFile="true".
>
> If you have a very data-heavy application (tons of bytes need to be
> sent back and forth to the client) or you have clients with very thin
> pipes (e.g. mobile), then compression might outweigh "efficiency" on
> the application server.
>

That pretty much sums it up, yea? ;)

I think the group consensus is to monitor the server CPU and use that as
the main factor in deciding whether to use compression or not.

I really appreciate your thoughts and the thoughts of the other responders.
I have a clear plan of attack now.


> - -chris
>
> > On Tue, Apr 18, 2017 at 9:08 AM, André Warnier (tomcat)
> > <aw@ice-sa.com> wrote:
> >
> >> On 18.04.2017 14:50, Chris Gamache wrote:
> >>
> >>> Using tomcat 8.0.43 ...
> >>>
> >>> I'm grappling with GZip compression options. Historically, I've
> >>> used a custom GZip filter and that's been fine for the most
> >>> part. If the file being served is under 50K the filter would
> >>> compress it in memory and send it out. If the file is over 50K,
> >>> it would connect the OutputStream to a GZipOutputStream rather
> >>> than compressing the whole thing in memory and then sending it
> >>> out from there. When that happens it doesn't send a
> >>> Content-Length header. This is fine for most browsers. Safari
> >>> has a problem with this and will decline to receive the file.
> >>> In looking at the GZip filter I've been using, it's kind-of
> >>> naive-- there must be a more intelligent compression option
> >>> built into tomcat by now, right?
> >>>
> >>> To enable compression, I set `compression="on"` in my
> >>> <Connector/> element in my server.xml file. There is on
> >>> sticking point-- if sendFile is available (asynchronous
> >>> file-sending by the DefaultServlet using NIO) it will trump
> >>> compression by default. I can turn off sendFile, and browsers
> >>> report that they are receiving compressed files. So it seems
> >>> like an all-or-nothing situation where compression and sendFile
> >>> are mutually exclusive. There are minimum file size settings
> >>> for both options, but no max file size settings so I can't say
> >>> "use sendFile for files under 50K and compression for files
> >>> above 50K" because sendFile will always trump compression.
> >>>
> >>> I think the idea of sending out static files asynchronously is
> >>> fantastic. I also want my pages to load faster by sending less
> >>> data.
> >>>
> >>> I figure the smart people who work on tomcat know a whole lot
> >>> more about this stuff than I do. They must have had a reason to
> >>> prioritize sendFile over compression, or the expert tomcat
> >>> administrators have figured out a way to balance the two.
> >>>
> >>> Maybe I've been staring at the problem too long, but I can't
> >>> figure it out.
> >>>
> >>> So, is it advisable turn of sendFile to engage compression? Or,
> >>> is there a combination of settings that will let me best use
> >>> them both?
> >>>
> >>>
> >> For what it's worth : sendfile() is a way by which the (web)
> >> application can just point the OS to a static file on disk, and
> >> say "send this". And the sendfile logic in the OS takes care of
> >> the rest, in the most efficient way possible for that OS, and the
> >> call returns ok to your application right away, even possibly
> >> before the sendfile() action has completed. The sticky point here
> >> is "a static file on disk". So if you want to send back a gzipped
> >> file, then the only solution is to first create that gzipped
> >> version as a file on disk, and then use sendfile() on that
> >> gzipped version. And then, presumably, you'd want to "clean up"
> >> these gzipped versions at some point. Which cannot happen right
> >> after you have called sendfile(), because you do not really know
> >> when it will be done actually sending the file. So it is not
> >> really a "priority" issue, it is that these are different things,
> >> and that sendfile() really only works on a file, not on dynamic
> >> output from a webapp.
> >>
> >>
> >>
> >>
> >> ---------------------------------------------------------------------
> >>
> >>
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> >> For additional commands, e-mail: users-help@tomcat.apache.org
> >>
> >>
> >
> -----BEGIN PGP SIGNATURE-----
> Comment: GPGTools - http://gpgtools.org
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQIcBAEBCAAGBQJY92V9AAoJEBzwKT+lPKRYx7IQAICSkzOhxFTF8BXQRivkS3Rb
> vfMvYSGnizdr80btMcfTNSWbu0SPh/TyYH4gfmeNUGLgFBdK0QrABCGLZih4470/
> lb57t7UwTYLCPvTrR/rXkgHzIkSIQtMbCj6q3To4GZvR2IRXUoYRnZXK5okUrv0/
> AWHUmqdflbjTKySFThFy/yMmtwo2tHhZY+DqlU0u1J7qPHhWWDcUkCBxCJUO1q9d
> +PkFnGzGWcQMtEBR46zxnARLMZemgI/x3nonCO0wtlABsdnJpsUdEmONrbBWdlx2
> 8TxJfRdZst6Au8n6qcdhGaqGg/7BqbBBgjHZKb6kebFc0lij0w2yCfhifYXcn/BC
> 2AZGB6GNxNZ96GCrbK53OoBTnNeLZBygumXBD65GJPsiF+hGjAiCy0ut3mT1o9aZ
> ROJ7KIY43U6PF7jqAAnmvvhdTZZGlP6rC2Ip7WBHTlLo8bKskCoXzYx0dqz6gyTV
> jBC2ausLIMWrO+NlXae01fPj9S5hZ7wzBtKORYNvfYa8US1JyUxoOwHiTcdJnnTP
> vNszZ3kLfcKZM5nKECN5foh4HV84uMcgrzf1bipFenghGzi6oa2rVJs2aWUaBofO
> DQudpKB0ri0e9ZQh2jZdSrNZwwKqLQMgy8kyDoZYcw6AQS5hFOoZleGxWV7E59hA
> ZBwjnoLIXiO3EAXbPAa5
> =dN1L
> -----END PGP SIGNATURE-----
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message