httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aaron Bannert <aa...@clove.org>
Subject Re: filtering huge request bodies (like 650MB files)
Date Wed, 10 Dec 2003 23:47:20 GMT
On Wed, Dec 10, 2003 at 06:29:28PM -0500, Glenn wrote:
> On Wed, Dec 10, 2003 at 03:18:44PM -0800, Stas Bekman wrote:
> > Are you saying that if I POST N MBytes of data to the server and just have 
> > the server send it back to me, it won't grow by that N MBytes of memory for 
> > the duration of that request? Can you pipe the data out as it comes in? I 
> > thought that you must read the data in before you can send it out (at least 
> > if it's the same client who sends and receives the data).
> 
> Well, in the case of CGI, mod_cgi and mod_cgid currently require that
> the CGI read in the entire body before mod_cgi(d?) will read the
> response from the CGI.  So a CGI "echo" program must buffer the whole
> response before mod_cgi(d?) will read the CGI output and send it back
> to the client.  If the CGI buffers to disk, no problem, but if the
> CGI buffers in memory, it will take a lot of memory (but not in Apache).
> 
> Obviously :-), that's a shortcoming of mod_cgi(d?), but might also be
> a problem for modules such as mod_php which preprocesses the CGI POST
> info before running the PHP script.

[slightly off-topic]

Actually, I believe that mod_cgi and mod_cgid are currently broken
WRT the CGI spec. The spec says that a CGI may read as much of an
incoming request body as it wishes and may return data as soon as
it wishes (all AIUI). That means that right now if you send a big
body to a CGI script that does not read the request body (which
is perfectly valid according to the CGI script) then mod_cgi[d] will
deadlock trying to write the rest of the body to the script.

The best way to fix this would be to support a poll()-like multiplexing
I/O scheme where data could be written do and read from the CGI process
at the same time. Unfortunately, that's not currently supported by
the Apache filter code.

-aaron

Mime
View raw message