httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bill Stoddard <b...@wstoddard.com>
Subject Re: filtering huge request bodies (like 650MB files)
Date Thu, 11 Dec 2003 01:01:52 GMT
Aaron Bannert wrote:

> On Wed, Dec 10, 2003 at 06:29:28PM -0500, Glenn wrote:
> 
>>On Wed, Dec 10, 2003 at 03:18:44PM -0800, Stas Bekman wrote:
>>
>>>Are you saying that if I POST N MBytes of data to the server and just have 
>>>the server send it back to me, it won't grow by that N MBytes of memory for 
>>>the duration of that request? Can you pipe the data out as it comes in? I 
>>>thought that you must read the data in before you can send it out (at least 
>>>if it's the same client who sends and receives the data).
>>
>>Well, in the case of CGI, mod_cgi and mod_cgid currently require that
>>the CGI read in the entire body before mod_cgi(d?) will read the
>>response from the CGI.  So a CGI "echo" program must buffer the whole
>>response before mod_cgi(d?) will read the CGI output and send it back
>>to the client.  If the CGI buffers to disk, no problem, but if the
>>CGI buffers in memory, it will take a lot of memory (but not in Apache).
>>
>>Obviously :-), that's a shortcoming of mod_cgi(d?), but might also be
>>a problem for modules such as mod_php which preprocesses the CGI POST
>>info before running the PHP script.
> 
> 
> [slightly off-topic]
> 
> Actually, I believe that mod_cgi and mod_cgid are currently broken
> WRT the CGI spec. The spec says that a CGI may read as much of an
> incoming request body as it wishes and may return data as soon as
> it wishes (all AIUI). That means that right now if you send a big
> body to a CGI script that does not read the request body (which
> is perfectly valid according to the CGI script) then mod_cgi[d] will
> deadlock trying to write the rest of the body to the script.
> 
> The best way to fix this would be to support a poll()-like multiplexing
> I/O scheme where data could be written do and read from the CGI process
> at the same time. Unfortunately, that's not currently supported by
> the Apache filter code.
> 
> -aaron

Interesting. Then Apache 1.3 is broken too. I believe Jeff posted a patch not too long ago
to enable full 
duplex interface between Apache and CGI scripts.

Bill



Mime
View raw message