tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From André Warnier (tomcat)>
Subject Re: sendFiles vs. compression
Date Wed, 19 Apr 2017 00:03:01 GMT
On 18.04.2017 20:03, Chris Gamache wrote:
> On Tue, Apr 18, 2017 at 11:24 AM, André Warnier (tomcat) <>
> wrote:
>> Hi again.
>> On this list, it is customary (and requested) to respond in-line and not
>> "top post".
>> See :, item #6.
>> It makes it easier to follow the conversation, as opposed to having to
>> scroll back and forth to find out what you are commenting on.
>> Noted. Apologies.
>> On 18.04.2017 16:58, Chris Gamache wrote:
>>> Excellent information. Thank you!
>>> Is there a way to create a split point where sendFile will handle files of
>>> certain mime types (or all mime-types except for an exclusion list of mime
>>> types) and/or of certain sizes while compression will handle files of
>>> other
>>> mime-types and/or certain sizes?
>>> Both settings have a minimum file size that engages their mechanism but to
>>> set up a division of labor I would think we would need all of
>>> include/exclude and max/min for both sendfile() and compression. Again, I
>>> could be missing something obvious by staring at the problem too long.
>> As you have probably already found out from the extensive and exquisitely
>> written on-line tomcat documentation, there do not seem to exist such
>> fine-tuned parameters available in the standard tomcat Connectors.
> Props. We could do better explaining the "why's" along with all the
> "what's". When I figure it out, I'll send a patch. ;) For now, I have to
> rely on my fine colleagues here in the list for some of the more intricate
> interpretations.
>> If you want this type of control, /and/ you can more or less determine the
>> kind of response you want to send by examining the request URL, I would
>> have a look at a servlet filter such as URLRewrite (
>>, which could examine the request and
>> determine to which specific response-generating servlet it should dispatch
>> the call. In that servlet you can then decide yourself to compress or not
>> the output, and/or to use sendfile.
>> So by rewriting the URL, the request gets re-evaluated such that a
> different <servlet-mapping /> would be engaged? I still have the dilemma of
> determining which ladle I use to serve the data with.
>>> @André and the rest of the listserv, In your opinions-- thinking about the
>>> web site consumer's experience, and having to choose either send
>>> sendfile()
>>> or compression-- is the juice worth the squeeze so-to-speak keeping
>>> sendfile() and sending uncompressed files, or is the better choice to
>>> enable compression at the expense of direct static file access and save
>>> bandwidth?
>> I think that this is a question to which you are the only one who can
>> provide the right answer, because "it depends" on a lot of factors that are
>> specific to your application, your mix of documents, your bandwidth
>> availability and cost, the requirements of your clients etc..
>> With all respect, and I do thank you for your willingness to respond-- if
> I had any frame of reference to base a decision on, I wouldn't have asked
> the question. Ask any front-end engineer what the single best thing to do
> to make a user's experience better when accessing a single-page web
> application, they will say "enable compression" so why it isn't turned on
> by default was a mystery, and that it plays second fiddle to serving static
> file from the file system in an efficient manner was a double mystery.

Your any front-end engineer either is a crook, or he wants to get rid of you quickly.
There is no such "one size fits all" answer, for such a vague question.

> Perhaps if my fellow tomcat users would share their thought processes in
> their particular situations for selecting one method over the other, that
> might help me look at my own situation and make a good decision.

You already got a glimpse of our thought processes in the matter :
We would start by trying out one configuration, in our own complex circumstances, and look

at the results. And if we identify one particular problem area and can then ask a more 
precise and focused question based on some collected facts, and we cannot answer it 
ourselves, we might post it to the tomcat user's list.

Alternatively, one of us might propose a dedicated paid consultancy, to inspect your 
application and your server and its connectivity and your user's usage pattern, and 
recommend a "best" solution based on collected facts, on his own experience in the matter,

and on your budget.

>>> On Tue, Apr 18, 2017 at 9:08 AM, André Warnier (tomcat) <>
>>> wrote:
>>> On 18.04.2017 14:50, Chris Gamache wrote:
>>>> Using tomcat 8.0.43 ...
>>>>> I'm grappling with GZip compression options. Historically, I've used
>>>>> custom GZip filter and that's been fine for the most part. If the file
>>>>> being served is under 50K the filter would compress it in memory and
>>>>> send
>>>>> it out. If the file is over 50K, it would connect the OutputStream to
>>>>> GZipOutputStream rather than compressing the whole thing in memory and
>>>>> then
>>>>> sending it out from there. When that happens it doesn't send a
>>>>> Content-Length header. This is fine for most browsers. Safari has a
>>>>> problem
>>>>> with this and will decline to receive the file. In looking at the GZip
>>>>> filter I've been using, it's kind-of naive-- there must be a more
>>>>> intelligent compression option built into tomcat by now, right?
>>>>> To enable compression, I set `compression="on"` in my <Connector/>
>>>>> element
>>>>> in my server.xml file. There is on sticking point-- if sendFile is
>>>>> available (asynchronous file-sending by the DefaultServlet using NIO)
>>>>> will trump compression by default. I can turn off sendFile, and browsers
>>>>> report that they are receiving compressed files. So it seems like an
>>>>> all-or-nothing situation where compression and sendFile are mutually
>>>>> exclusive. There are minimum file size settings for both options, but
>>>>> max file size settings so I can't say "use sendFile for files under 50K
>>>>> and
>>>>> compression for files above 50K" because sendFile will always trump
>>>>> compression.
>>>>> I think the idea of sending out static files asynchronously is
>>>>> fantastic..
>>>>> I
>>>>> also want my pages to load faster by sending less data.
>>>>> I figure the smart people who work on tomcat know a whole lot more about
>>>>> this stuff than I do. They must have had a reason to prioritize sendFile
>>>>> over compression, or the expert tomcat administrators have figured out
>>>>> way to balance the two.
>>>>> Maybe I've been staring at the problem too long, but I can't figure it
>>>>> out.
>>>>> So, is it advisable turn of sendFile to engage compression? Or, is
>>>>> there a
>>>>> combination of settings that will let me best use them both?
>>>>> For what it's worth :
>>>> sendfile() is a way by which the (web) application can just point the OS
>>>> to a static file on disk, and say "send this". And the sendfile logic in
>>>> the OS takes care of the rest, in the most efficient way possible for
>>>> that
>>>> OS, and the call returns ok to your application right away, even possibly
>>>> before the sendfile() action has completed.
>>>> The sticky point here is "a static file on disk".
>>>> So if you want to send back a gzipped file, then the only solution is to
>>>> first create that gzipped version as a file on disk, and then use
>>>> sendfile() on that gzipped version.
>>>> And then, presumably, you'd want to "clean up" these gzipped versions at
>>>> some point.
>>>> Which cannot happen right after you have called sendfile(), because you
>>>> do
>>>> not really know when it will be done actually sending the file.
>>>> So it is not really a "priority" issue, it is that these are different
>>>> things, and that sendfile() really only works on a file, not on dynamic
>>>> output from a webapp.
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail:
>>>> For additional commands, e-mail:
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail:
>> For additional commands, e-mail:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message