httpd-cvs mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From minf...@apache.org
Subject svn commit: r592951 - in /httpd/httpd/trunk: ./ docs/manual/mod/ include/ modules/filters/ modules/http/ server/
Date Wed, 07 Nov 2007 23:31:10 GMT
Author: minfrin
Date: Wed Nov  7 15:31:03 2007
New Revision: 592951

URL: http://svn.apache.org/viewvc?rev=592951&view=rev
Log:
core: Add the option to keep aside a request body up to a certain
size that would otherwise be discarded, to be consumed by filters
such as mod_include. When enabled for a directory, POST requests
to shtml files can be passed through to embedded scripts as POST
requests, rather being downgraded to GET requests.

Modified:
    httpd/httpd/trunk/CHANGES
    httpd/httpd/trunk/docs/manual/mod/core.xml
    httpd/httpd/trunk/docs/manual/mod/mod_include.xml
    httpd/httpd/trunk/include/ap_mmn.h
    httpd/httpd/trunk/include/httpd.h
    httpd/httpd/trunk/modules/filters/mod_include.c
    httpd/httpd/trunk/modules/http/http_core.c
    httpd/httpd/trunk/modules/http/http_filters.c
    httpd/httpd/trunk/modules/http/mod_core.h
    httpd/httpd/trunk/server/request.c

Modified: httpd/httpd/trunk/CHANGES
URL: http://svn.apache.org/viewvc/httpd/httpd/trunk/CHANGES?rev=592951&r1=592950&r2=592951&view=diff
==============================================================================
--- httpd/httpd/trunk/CHANGES [utf-8] (original)
+++ httpd/httpd/trunk/CHANGES [utf-8] Wed Nov  7 15:31:03 2007
@@ -2,6 +2,12 @@
 Changes with Apache 2.3.0
 [ When backported to 2.2.x, remove entry from this file ]
 
+  *) core: Add the option to keep aside a request body up to a certain
+     size that would otherwise be discarded, to be consumed by filters
+     such as mod_include. When enabled for a directory, POST requests
+     to shtml files can be passed through to embedded scripts as POST
+     requests, rather being downgraded to GET requests. [Graham Leggett]
+
   *) mod_charset_lite: Don't crash when the request has no associated
      filename.  [Jeff Trawick]
 

Modified: httpd/httpd/trunk/docs/manual/mod/core.xml
URL: http://svn.apache.org/viewvc/httpd/httpd/trunk/docs/manual/mod/core.xml?rev=592951&r1=592950&r2=592951&view=diff
==============================================================================
--- httpd/httpd/trunk/docs/manual/mod/core.xml (original)
+++ httpd/httpd/trunk/docs/manual/mod/core.xml Wed Nov  7 15:31:03 2007
@@ -1504,6 +1504,63 @@
 </directivesynopsis>
 
 <directivesynopsis>
+<name>KeptBodySize</name>
+<description>Keep the request body instead of discarding it up to
+the specified maximum size, for potential use by filters such as
+mod_include.</description>
+<syntax>KeptBodySize <var>maximum size in bytes</var></syntax>
+<default>KeptBodySize 0</default>
+<contextlist><context>directory</context>
+</contextlist>
+
+<usage>
+    <p>Under normal circumstances, request handlers such as the
+    default handler for static files will discard the request body
+    when it is not needed by the request handler. As a result,
+    filters such as mod_include are limited to making <code>GET</code> requests
+    only when including other URLs as subrequests, even if the
+    original request was a <code>POST</code> request, as the discarded
+    request body is no longer available once filter processing is
+    taking place.</p>
+
+    <p>When this directive has a value greater than zero, request
+    handlers that would otherwise discard request bodies will
+    instead set the request body aside for use by filters up to
+    the maximum size specified. In the case of the mod_include
+    filter, an attempt to <code>POST</code> a request to the static
+    shtml file will cause any subrequests to be <code>POST</code>
+    requests, instead of <code>GET</code> requests as before.</p>
+
+    <p>This feature makes it possible to break up complex web pages and
+    web applications into small individual components, and combine
+    the components and the surrounding web page structure together
+    using <module>mod_include</module>. The components can take the
+    form of CGI programs, scripted languages, or URLs reverse proxied
+    into the URL space from another server using
+    <module>mod_proxy</module>.</p>
+
+    <p><strong>Note:</strong> Each request set aside has to be set
+    aside in temporary RAM until the request is complete. As a result,
+    care should be taken to ensure sufficient RAM is available on the
+    server to support the intended load. Use of this directive
+    should be limited to where needed on targeted parts of your
+    URL space, and with the lowest possible value that is still big
+    enough to hold a request body.</p>
+
+    <p>If the request size sent by the client exceeds the maximum
+    size allocated by this directive, the server will return
+    <code>413 Request Entity Too Large</code>.</p>
+
+    <p>Handlers such as <module>mod_cgi</module> that consume request
+    bodies for their own purposes rather than discard them do not take
+    this directive into account.</p>
+
+</usage>
+
+<seealso><a href="mod_include.html">mod_include</a> documentation</seealso>
+</directivesynopsis>
+
+<directivesynopsis>
 <name>KeepAliveTimeout</name>
 <description>Amount of time the server will wait for subsequent
 requests on a persistent connection</description>

Modified: httpd/httpd/trunk/docs/manual/mod/mod_include.xml
URL: http://svn.apache.org/viewvc/httpd/httpd/trunk/docs/manual/mod/mod_include.xml?rev=592951&r1=592950&r2=592951&view=diff
==============================================================================
--- httpd/httpd/trunk/docs/manual/mod/mod_include.xml (original)
+++ httpd/httpd/trunk/docs/manual/mod/mod_include.xml Wed Nov  7 15:31:03 2007
@@ -343,14 +343,22 @@
       <p>If the specified URL is a CGI program, the program will be
       executed and its output inserted in place of the directive in the
       parsed file. You may include a query string in a CGI url:</p>
-     
+ 
       <example>
         &lt;!--#include virtual="/cgi-bin/example.cgi?argument=value" --&gt;
       </example>
-           
+ 
       <p><code>include virtual</code> should be used in preference
       to <code>exec cgi</code> to include the output of CGI programs
       into an HTML document.</p>
+
+      <p>If the <directive module="core">KeptBodySize</directive>
+      directive is correctly configured and valid for this included
+      file, attempts to POST requests to the enclosing HTML document
+      will be passed through to subrequests as POST requests as well.
+      Without the directive, all subrequests are processed as GET
+      requests.</p>
+
       </dd>
       </dl>
     </section> <!-- /include -->

Modified: httpd/httpd/trunk/include/ap_mmn.h
URL: http://svn.apache.org/viewvc/httpd/httpd/trunk/include/ap_mmn.h?rev=592951&r1=592950&r2=592951&view=diff
==============================================================================
--- httpd/httpd/trunk/include/ap_mmn.h (original)
+++ httpd/httpd/trunk/include/ap_mmn.h Wed Nov  7 15:31:03 2007
@@ -139,14 +139,15 @@
  * 20071023.2 (2.3.0-dev)  Add ap_mod_status_reqtail
  * 20071023.3 (2.3.0-dev)  Declare ap_time_process_request() as part of the
  *                         public scoreboard API.
+ * 20071108.1 (2.3.0-dev)  Add the optional kept_body brigade to request_rec
  */
 
 #define MODULE_MAGIC_COOKIE 0x41503234UL /* "AP24" */
 
 #ifndef MODULE_MAGIC_NUMBER_MAJOR
-#define MODULE_MAGIC_NUMBER_MAJOR 20071023
+#define MODULE_MAGIC_NUMBER_MAJOR 20071108
 #endif
-#define MODULE_MAGIC_NUMBER_MINOR 3                    /* 0...n */
+#define MODULE_MAGIC_NUMBER_MINOR 1                    /* 0...n */
 
 /**
  * Determine if the server's current MODULE_MAGIC_NUMBER is at least a

Modified: httpd/httpd/trunk/include/httpd.h
URL: http://svn.apache.org/viewvc/httpd/httpd/trunk/include/httpd.h?rev=592951&r1=592950&r2=592951&view=diff
==============================================================================
--- httpd/httpd/trunk/include/httpd.h (original)
+++ httpd/httpd/trunk/include/httpd.h Wed Nov  7 15:31:03 2007
@@ -988,6 +988,9 @@
     /** A flag to determine if the eos bucket has been sent yet */
     int eos_sent;
 
+    /** The optional kept body of the request. */
+    apr_bucket_brigade *kept_body;
+
 /* Things placed at the end of the record to avoid breaking binary
  * compatibility.  It would be nice to remember to reorder the entire
  * record to improve 64bit alignment the next time we need to break

Modified: httpd/httpd/trunk/modules/filters/mod_include.c
URL: http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/filters/mod_include.c?rev=592951&r1=592950&r2=592951&view=diff
==============================================================================
--- httpd/httpd/trunk/modules/filters/mod_include.c (original)
+++ httpd/httpd/trunk/modules/filters/mod_include.c Wed Nov  7 15:31:03 2007
@@ -1712,7 +1712,12 @@
             }
         }
         else {
-            rr = ap_sub_req_lookup_uri(parsed_string, r, f->next);
+            if (r->kept_body) {
+                rr = ap_sub_req_method_uri(r->method, parsed_string, r, f->next);
+            }
+            else {
+                rr = ap_sub_req_lookup_uri(parsed_string, r, f->next);
+            }
         }
 
         if (!error_fmt && rr->status != HTTP_OK) {

Modified: httpd/httpd/trunk/modules/http/http_core.c
URL: http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/http/http_core.c?rev=592951&r1=592950&r2=592951&view=diff
==============================================================================
--- httpd/httpd/trunk/modules/http/http_core.c (original)
+++ httpd/httpd/trunk/modules/http/http_core.c Wed Nov  7 15:31:03 2007
@@ -37,6 +37,7 @@
 
 /* Handles for core filters */
 AP_DECLARE_DATA ap_filter_rec_t *ap_http_input_filter_handle;
+AP_DECLARE_DATA ap_filter_rec_t *ap_kept_body_input_filter_handle;
 AP_DECLARE_DATA ap_filter_rec_t *ap_http_header_filter_handle;
 AP_DECLARE_DATA ap_filter_rec_t *ap_chunk_filter_handle;
 AP_DECLARE_DATA ap_filter_rec_t *ap_http_outerror_filter_handle;
@@ -89,6 +90,20 @@
     return NULL;
 }
 
+static const char *set_kept_body_size(cmd_parms *cmd, void *dconf,
+                                      const char *arg)
+{
+    core_dir_conf *conf = dconf;
+
+    if (APR_SUCCESS != apr_strtoff(&(conf->keep_body), arg, NULL, 0)
+        || conf->keep_body < 0) {
+        return "KeptBodySize must be a size in bytes, or zero.";
+    }
+    conf->keep_body_set = 1;
+
+    return NULL;
+}
+
 static const command_rec http_cmds[] = {
     AP_INIT_TAKE1("KeepAliveTimeout", set_keep_alive_timeout, NULL, RSRC_CONF,
                   "Keep-Alive timeout duration (sec)"),
@@ -97,6 +112,8 @@
                   "or 0 for infinite"),
     AP_INIT_TAKE1("KeepAlive", set_keep_alive, NULL, RSRC_CONF,
                   "Whether persistent connections should be On or Off"),
+    AP_INIT_TAKE1("KeptBodySize", set_kept_body_size, NULL, ACCESS_CONF,
+                  "Maximum size of request bodies kept aside for use by filters"),
     { NULL }
 };
 
@@ -268,6 +285,9 @@
     ap_http_input_filter_handle =
         ap_register_input_filter("HTTP_IN", ap_http_filter,
                                  NULL, AP_FTYPE_PROTOCOL);
+    ap_kept_body_input_filter_handle =
+        ap_register_input_filter("KEPT_BODY", ap_kept_body_filter,
+                                 ap_kept_body_filter_init, AP_FTYPE_RESOURCE);
     ap_http_header_filter_handle =
         ap_register_output_filter("HTTP_HEADER", ap_http_header_filter,
                                   NULL, AP_FTYPE_PROTOCOL);
@@ -283,12 +303,35 @@
     ap_method_registry_init(p);
 }
 
+static void *create_core_dir_config(apr_pool_t *p, char *dummy)
+{
+    core_dir_conf *new =
+        (core_dir_conf *) apr_pcalloc(p, sizeof(core_dir_conf));
+
+    new->keep_body_set = 0; /* unset */
+    new->keep_body = 0; /* don't by default */
+
+    return (void *) new;
+}
+
+static void *merge_core_dir_config(apr_pool_t *p, void *basev, void *addv)
+{
+    core_dir_conf *new = (core_dir_conf *) apr_pcalloc(p, sizeof(core_dir_conf));
+    core_dir_conf *add = (core_dir_conf *) addv;
+    core_dir_conf *base = (core_dir_conf *) basev;
+
+    new->keep_body = (add->keep_body_set == 0) ? base->keep_body : add->keep_body;
+    new->keep_body_set = add->keep_body_set || base->keep_body_set;
+
+    return new;
+}
+
 module AP_MODULE_DECLARE_DATA http_module = {
     STANDARD20_MODULE_STUFF,
-    NULL,              /* create per-directory config structure */
-    NULL,              /* merge per-directory config structures */
-    NULL,              /* create per-server config structure */
-    NULL,              /* merge per-server config structures */
-    http_cmds,         /* command apr_table_t */
-    register_hooks     /* register hooks */
+    create_core_dir_config, /* create per-directory config structure */
+    merge_core_dir_config,  /* merge per-directory config structures */
+    NULL,                   /* create per-server config structure */
+    NULL,                   /* merge per-server config structures */
+    http_cmds,              /* command apr_table_t */
+    register_hooks          /* register hooks */
 };

Modified: httpd/httpd/trunk/modules/http/http_filters.c
URL: http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/http/http_filters.c?rev=592951&r1=592950&r2=592951&view=diff
==============================================================================
--- httpd/httpd/trunk/modules/http/http_filters.c (original)
+++ httpd/httpd/trunk/modules/http/http_filters.c Wed Nov  7 15:31:03 2007
@@ -55,6 +55,8 @@
 #include <unistd.h>
 #endif
 
+extern module AP_MODULE_DECLARE_DATA http_module;
+
 static long get_chunk_size(char *);
 
 typedef struct http_filter_ctx {
@@ -1145,8 +1147,11 @@
  */
 AP_DECLARE(int) ap_discard_request_body(request_rec *r)
 {
-    apr_bucket_brigade *bb;
+    apr_bucket_brigade *bb, *kept_body = NULL;
+    apr_bucket *e;
     int rv, seen_eos;
+    core_dir_conf *dconf;
+    apr_size_t left = 0;
 
     /* Sometimes we'll get in a state where the input handling has
      * detected an error where we want to drop the connection, so if
@@ -1160,6 +1165,20 @@
         return OK;
     }
 
+    /* We may want to save this body away if the administrator has
+     * asked us to do so for this directory. This allows the body
+     * to be reexamined by filters such as mod_include, even though
+     * the main request has no need for this body.
+     */
+    if (!r->kept_body) {
+        dconf = ap_get_module_config(r->per_dir_config,
+                                     &http_module);
+        if (dconf->keep_body > 0) {
+            left = dconf->keep_body;
+            kept_body = apr_brigade_create(r->pool, r->connection->bucket_alloc);
+        }
+    }
+
     bb = apr_brigade_create(r->pool, r->connection->bucket_alloc);
     seen_eos = 0;
     do {
@@ -1211,10 +1230,33 @@
                 apr_brigade_destroy(bb);
                 return HTTP_BAD_REQUEST;
             }
+
+            /* If we have been asked to, keep the data up until the
+             * configured limit. If the limit is exceeded, we return an
+             * HTTP_REQUEST_ENTITY_TOO_LARGE response so the caller is
+             * clear the server couldn't handle their request.
+             */
+            if (kept_body) {
+                if (len <= left) {
+                    apr_bucket_copy(bucket, &e);
+                    APR_BRIGADE_INSERT_TAIL(kept_body, e);
+                    left -= len;
+                }
+                else {
+                    apr_brigade_destroy(bb);
+                    apr_brigade_destroy(kept_body);
+                    return HTTP_REQUEST_ENTITY_TOO_LARGE;
+                }
+            }
+            
         }
         apr_brigade_cleanup(bb);
     } while (!seen_eos);
 
+    if (kept_body) {
+        r->kept_body = kept_body;
+    }
+
     return OK;
 }
 
@@ -1421,3 +1463,101 @@
     return ap_pass_brigade(f->next,  b);
 }
 
+typedef struct kept_body_filter_ctx {
+    apr_off_t offset;
+    apr_off_t remaining;
+} kept_body_ctx_t;
+
+/**
+ * Initialisation of filter to handle a kept body on subrequests.
+ * 
+ * If a body is to be reinserted into a subrequest, any chunking will have
+ * been removed from the body during storage. We need to change the request
+ * from Transfer-Encoding: chunked to an explicit Content-Length.
+ */
+int ap_kept_body_filter_init(ap_filter_t *f) {
+    apr_off_t length = 0;
+    request_rec *r = f->r;
+    apr_bucket_brigade *kept_body = r->kept_body;
+
+    if (kept_body) {
+        apr_table_unset(r->headers_in, "Transfer-Encoding");
+        apr_brigade_length(kept_body, 1, &length);
+        apr_table_set(r->headers_in, "Content-Length", apr_off_t_toa(r->pool, length));
+    }
+
+    return OK;
+}
+
+/**
+ * Filter to handle a kept body on subrequests.
+ * 
+ * If a body has been previously kept by the request, and if a subrequest wants
+ * to re-insert the body into the request, this input filter makes it happen.
+ */
+apr_status_t ap_kept_body_filter(ap_filter_t *f, apr_bucket_brigade *b,
+                                 ap_input_mode_t mode, apr_read_type_e block,
+                                 apr_off_t readbytes) {
+    request_rec *r = f->r;
+    apr_bucket_brigade *kept_body = r->kept_body;
+    kept_body_ctx_t *ctx = f->ctx;
+    apr_bucket *ec, *e2;
+    apr_status_t rv;
+
+    /* just get out of the way of things we don't want. */
+    if (!kept_body || (mode != AP_MODE_READBYTES && mode != AP_MODE_GETLINE)) {
+        return ap_get_brigade(f->next, b, mode, block, readbytes);
+    }
+
+    /* set up the context if it does not already exist */
+    if (!ctx) {
+        f->ctx = ctx = apr_palloc(f->r->pool, sizeof(*ctx));
+        ctx->offset = 0;
+        apr_brigade_length(kept_body, 1, &ctx->remaining);
+    }
+
+    /* kept_body is finished, send next filter */
+    if (ctx->remaining <= 0) {
+        return ap_get_brigade(f->next, b, mode, block, readbytes);
+    }
+
+    /* send all of the kept_body, but no more */
+    if (readbytes > ctx->remaining) {
+        readbytes = ctx->remaining;
+    }
+
+    /* send part of the kept_body */
+    if ((rv = apr_brigade_partition(kept_body, ctx->offset, &ec)) != APR_SUCCESS)
{
+        ap_log_rerror(APLOG_MARK, APLOG_ERR, rv, r,
+                      "apr_brigade_partition() failed on kept_body at %" APR_OFF_T_FMT, ctx->offset);
+        return rv;
+    }
+    if ((rv = apr_brigade_partition(kept_body, ctx->offset + readbytes, &e2)) != APR_SUCCESS)
{
+        ap_log_rerror(APLOG_MARK, APLOG_ERR, rv, r,
+                      "apr_brigade_partition() failed on kept_body at %" APR_OFF_T_FMT, ctx->offset
+ readbytes);
+        return rv;
+    }
+ 
+    do {
+        apr_bucket *foo;
+        const char *str;
+        apr_size_t len;
+
+        if (apr_bucket_copy(ec, &foo) != APR_SUCCESS) {
+            /* As above; this should not fail since the bucket has
+             * a known length, but just to be sure, this takes
+             * care of uncopyable buckets that do somehow manage
+             * to slip through.  */
+            /* XXX: check for failure? */
+            apr_bucket_read(ec, &str, &len, APR_BLOCK_READ);
+            apr_bucket_copy(ec, &foo);
+        }
+        APR_BRIGADE_INSERT_TAIL(b, foo);
+        ec = APR_BUCKET_NEXT(ec);
+    } while (ec != e2);
+
+    ctx->remaining -= readbytes;
+    ctx->offset += readbytes;
+    return APR_SUCCESS;
+
+}

Modified: httpd/httpd/trunk/modules/http/mod_core.h
URL: http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/http/mod_core.h?rev=592951&r1=592950&r2=592951&view=diff
==============================================================================
--- httpd/httpd/trunk/modules/http/mod_core.h (original)
+++ httpd/httpd/trunk/modules/http/mod_core.h Wed Nov  7 15:31:03 2007
@@ -38,8 +38,17 @@
 extern "C" {
 #endif
 
+/**
+ * Core per-directory configuration.
+ */
+typedef struct {
+    apr_off_t keep_body;
+    int keep_body_set;
+} core_dir_conf;
+
 /* Handles for core filters */
 extern AP_DECLARE_DATA ap_filter_rec_t *ap_http_input_filter_handle;
+extern AP_DECLARE_DATA ap_filter_rec_t *ap_kept_body_input_filter_handle;
 extern AP_DECLARE_DATA ap_filter_rec_t *ap_http_header_filter_handle;
 extern AP_DECLARE_DATA ap_filter_rec_t *ap_chunk_filter_handle;
 extern AP_DECLARE_DATA ap_filter_rec_t *ap_http_outerror_filter_handle;
@@ -51,6 +60,14 @@
 apr_status_t ap_http_filter(ap_filter_t *f, apr_bucket_brigade *b,
                             ap_input_mode_t mode, apr_read_type_e block,
                             apr_off_t readbytes);
+
+/* Filter to handle a kept body on subrequests */
+apr_status_t ap_kept_body_filter(ap_filter_t *f, apr_bucket_brigade *b,
+                                 ap_input_mode_t mode, apr_read_type_e block,
+                                 apr_off_t readbytes);
+
+/* Initialisation of filter to handle a kept body on subrequests */
+int ap_kept_body_filter_init(ap_filter_t *f);
 
 /* HTTP/1.1 chunked transfer encoding filter. */
 apr_status_t ap_http_chunk_filter(ap_filter_t *f, apr_bucket_brigade *b);

Modified: httpd/httpd/trunk/server/request.c
URL: http://svn.apache.org/viewvc/httpd/httpd/trunk/server/request.c?rev=592951&r1=592950&r2=592951&view=diff
==============================================================================
--- httpd/httpd/trunk/server/request.c (original)
+++ httpd/httpd/trunk/server/request.c Wed Nov  7 15:31:03 2007
@@ -1566,6 +1566,16 @@
      * until some module interjects and changes the value.
      */
     rnew->used_path_info = AP_REQ_DEFAULT_PATH_INFO;
+    
+    /* Pass on the kept body (if any) into the new request. */
+    rnew->kept_body = r->kept_body;
+
+    /*
+     * Add the KEPT_BODY filter, which will insert any body marked to be
+     * kept for the use of a subrequest, into the subrequest.
+     */
+    ap_add_input_filter_handle(ap_kept_body_input_filter_handle,
+                               NULL, rnew, rnew->connection);
 
     return rnew;
 }



Mime
View raw message