From new-httpd-return-17211-apmail-new-httpd-archive=apache.org@apache.org Mon Jul 02 18:55:21 2001 Return-Path: Delivered-To: apmail-new-httpd-archive@apache.org Received: (qmail 88416 invoked by uid 500); 2 Jul 2001 18:55:18 -0000 Mailing-List: contact new-httpd-help@apache.org; run by ezmlm Precedence: bulk Reply-To: new-httpd@apache.org list-help: list-unsubscribe: list-post: Delivered-To: mailing list new-httpd@apache.org Received: (qmail 88399 invoked from network); 2 Jul 2001 18:55:17 -0000 Message-ID: <01d801c10329$85c18800$85431b09@sashimi> From: "Bill Stoddard" To: References: Subject: Re: cvs commit: httpd-2.0 STATUS Date: Mon, 2 Jul 2001 15:02:20 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 X-Spam-Rating: h31.sny.collab.net 1.6.2 0/1000/N > On Mon, 2 Jul 2001, Bill Stoddard wrote: > > > > > > > >cgi on all platforms is broken. Specifically, with the addition of > > > >filters, we have lost the ability to flush partially written buffers > > > >received from CGI scripts to the network. > > > [...] > > > >Apache 2.0 always does a blocking read (in the content length filter) > > > > > > Partial writes to the network from a CGI and content length are > > > mutually exclusive. > > > > > > > No S**t!! :-) That's why it's broken! > > > > Working on a fix now. > > > Bill, there is no fix for this. If the content_length filter determines > that a C-L is required, then we can't stream CGI's. It looks like the > problem is simply that we are using the wrong options to apr_bucket_read. > Switch that to a APR_NONBLOCK_READ, and the problem should go away. > > Ryan I don't think changing the read in the content_length filter to nonblocking is the right solution to this problem because we still need to force the network flush. I will investigate your suggestion though because I might be overlooking something. IMO the decision to flush or not and how to read from the pipe should be entirely controlled by mod_cgi. We need a function similar to the old ap_send_fb() code. To demonstrate, here is some code. YES YES YES I know we don't want to commit this as written. This code will cause a data copies of all bytes off the stack buffer and into a heap buffer. Making this zero copy is easy enough, just wanted to post what I have been playing with the last 30 minutes to generally show how I think we should solve this problem (ie, control the byte stream at the top of the filter chain rather than deep in the filter chain). I also want to play with making the interface between Apache and CGI scripts full duplex as well (to allow a CGI script to read a large POST request and begin responding to that POST immediately) which will require controlling the byte stream at the top of the filter stack as well. Bill Index: mod_cgi.c =================================================================== RCS file: /home/cvs/httpd-2.0/modules/generators/mod_cgi.c,v retrieving revision 1.92 diff -u -u -r1.92 mod_cgi.c --- mod_cgi.c 2001/02/28 15:24:05 1.92 +++ mod_cgi.c 2001/07/02 18:26:53 @@ -768,16 +768,43 @@ } if (!r->header_only) { + #define IOBUFSIZE 8192 + apr_status_t rv; + char buf[IOBUFSIZE]; + int len = IOBUFSIZE; + apr_interval_time_t timeout; + bb = apr_brigade_create(r->pool); - b = apr_bucket_pipe_create(script_in); - APR_BRIGADE_INSERT_TAIL(bb, b); - b = apr_bucket_eos_create(); - APR_BRIGADE_INSERT_TAIL(bb, b); - ap_pass_brigade(r->output_filters, bb); - } + + /* Set the pipe to non-blocking for the first read */ + apr_file_pipe_timeout_get(script_in, &timeout); + apr_file_pipe_timeout_set(script_in, 0); - log_script_err(r, script_err); - apr_file_close(script_err); + while (!r->connection->aborted) { + len = IOBUFSIZE; + rv = apr_file_read(script_in, buf, &len); + if (rv == APR_EOF || (rv == APR_SUCCESS && len == 0)) { + b = apr_bucket_eos_create(); + APR_BRIGADE_INSERT_TAIL(bb, b); + ap_pass_brigade(r->output_filters, bb); + break; + } + else if (rv == APR_EAGAIN) { + /* Set the pipe to blocking and flush the output stream. */ + apr_file_pipe_timeout_set(script_in, timeout); + b = apr_bucket_flush_create(); + APR_BRIGADE_INSERT_TAIL(bb, b); + ap_pass_brigade(r->output_filters, bb); + } + else { + b = apr_bucket_transient_create(buf, len); + APR_BRIGADE_INSERT_TAIL(bb, b); + ap_pass_brigade(r->output_filters, bb); + } + } + log_script_err(r, script_err); + apr_file_close(script_err); + } } if (script_in && nph) {