httpd-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nadav Har'El <>
Subject Re: [users@httpd] mod-cgi reads entire output into memory...
Date Tue, 17 Jul 2012 20:25:09 GMT
On Tue, Jul 17, 2012, Nadav Har'El wrote about "Re: [users@httpd] mod-cgi reads entire output
into memory...":
> I "almost" have such code, but ran into a mystery - where in the
> request_req can I find the client socket, so I can write to it directly?
> There are so many layers of output filters, APR, etc., that I can't seem
> to find this simple thing...

I'm answering myself ;-)

It turns out that ap_get_conn_socket(r->connection) extracts the APR
socket object from the connection, and then apr_os_sock_get() gets the
file-descriptor out of the this APR object, so I can have the child
process inherit it. One thing that took me a while to notice is that
apache sets this socket to be nonblocking - so I need to make it
blocking again (with fcntl(fd, F_SETFL,0)) to make it usable as the
CGI program's standard output.

The resulting code is ugly, hairy and Unix-only, but it works and
performance is significantly better than normal mod_cgi for my usecase
of very large program-generated files. For example, a simple CGI
generating 512 MB of data took 0.9 seconds to fetch locally when using
mod_cgi, but only 0.4 seconds with my "zero-copy mod_cgi".

Additionally, after this local fetch with my "zero-copy mod_cgi", the
httpd process remains tiny (2 MB resident), while with the original
"pass through the do-nothing bucket brigade" mod_cgi, httpd grew to
86 MB resident.

I'm still wondering what's the justification of mod_cgi always reading
(from a pipe) and writing again (to the socket) the output of the
program, even when there are no output filters and even when we have an
NPH cgi (a CGI which generates in its own headers). Is it just that
nobody ever cared about this usecase (and in general, mod_cgi went out
of fashion), or is there a genuine reason why it's important for Apache
to read the output and write it back to the socket?


Nadav Har'El                        |     Tuesday, Jul 17 2012, 28 Tammuz 5772             |-----------------------------------------
Phone +972-523-790466, ICQ 13349191 |Some people bring happiness wherever they           |go, others whenever they go.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message