httpd-bugs mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bugzi...@apache.org
Subject DO NOT REPLY [Bug 52595] New: requests with gzip+chunked encoded body don't proxy reliably
Date Fri, 03 Feb 2012 19:57:33 GMT
https://issues.apache.org/bugzilla/show_bug.cgi?id=52595

             Bug #: 52595
           Summary: requests with gzip+chunked encoded body don't proxy
                    reliably
           Product: Apache httpd-2
           Version: 2.2.21
          Platform: PC
        OS/Version: Linux
            Status: NEW
          Severity: normal
          Priority: P2
         Component: mod_deflate
        AssignedTo: bugs@httpd.apache.org
        ReportedBy: lmeyer@vmware.com
    Classification: Unclassified


Created attachment 28258
  --> https://issues.apache.org/bugzilla/attachment.cgi?id=28258
Tools for reproducing the problem - script, test load, config, backend JSP

When request bodies are posted with gzip compression + chunked transfer, then
proxied to a Tomcat backend, they are subject to sporadic failures with all
three proxy methods I've tried: mod_proxy_http, mod_proxy_ajp, and mod_jk.

The problem does not occur without gzip. The failures are similar enough that I
suspect the problem has to do with the hand-off from mod_deflate, though it
could be further along in the request handling. It does not succeed or fail the
same each request even when exactly the same content is sent with exactly the
same chunk sizes, so I suspect that network timing and/or internal state may
play a role.

This may be related to https://issues.apache.org/bugzilla/show_bug.cgi?id=52492
- then again, it may not. It seemed different enough to me to merit a separate
issue.

This problem came from a real-world system issue that initially I suspected was
an AJP problem. When transferred via mod_jk, the request body would
sporadically be truncated when it was received by the backend (as confirmed by
packet trace). Aside from that all appears fine.

With mod_proxy_ajp, this use case never works at all that I have seen. The
entire body may be transmitted, but instead of sending the finishing 0-size
chunk, httpd closes the proxy connection and returns a 200 to the client (with
empty body). Errors in the log indicate mod_proxy_ajp.c(382): (20014)Internal
error: ap_get_brigade failed

With mod_proxy_http, this manifests as a 0-size chunk coming in the middle of
the request transfer. The backend interprets this as the end of the request
(which is normally what the 0-size chunk means) and responds normally to that
truncated request. Then when the proxy continues sending more chunks, the
backend returns a 400 and closes the connection. The response to the client
depends on timing - either the 200 response from the backend is returned, or
(if the connection is closed before the request is fully sent) a 502 error is
returned.

Side note: mod_proxy_http passes along the Content-Encoding: gzip header to the
backend even though the traffic is not compressed at that point. This is
probably a bug and might cause problems with some backends.

In order to test all this, I put together a Perl script that can gzip and chunk
a request allowing me to control chunk sizes (I found that some patterns made
this occur more often than others). I also created a JSP for the backend that
counts the incoming bytes and returns the count in the response so that the
script can verify whether the full content made it through. The content is
arbitrary, but for my tests, I chose a small portion of the system dictionary.
These are all attached. 

I will further attach logs and packet traces recording failures with each proxy
method. These are against 2.2.21, but I have observed the same results with
other versions (2.2.22, 2.2.17, 2.3.16). Again, these can be very sporadic, so
when reproducing, many requests may be required before anything goes wrong (the
script automates this). You may see many successes or many failures in a row.
It would not surprise me if this varied by OS, libz version, architecture, etc.
but I know it's not specific to my test system. In general, a good way to
trigger the problem is the "randub" algorithm with a small chunk size (like 100
to send chunks with random sizes between 100 and 200).

My tests have been on Linux (CentOS 5).
$ uname -a
Linux 2.6.18-238.19.1.el5 #1 SMP Fri Jul 15 07:32:29 EDT 2011 i686 i686 i386
GNU/Linux
$ rpm -q zlib
zlib-1.2.3-4.el5

Thanks for taking a look at this.

-- 
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org


Mime
View raw message