httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From r..@covalent.net
Subject Filtered I/O ... again. :-)
Date Wed, 31 May 2000 15:55:10 GMT

This is my latest attempt at being able to filter I/O.  Notice I say
filter not layer.  I do not consider this a full blown Layered I/O
implementation, but it is a relatively clean filtering approach.  This
basically gives us 85-90% of what a full blown layered approach would
without having to implement the whole thing.  So, I'm going to outline the
approach, and then some improvements that need to be made to this code,
and then the patch is at the end.  If this patch gets accepted, the docs
will be committed with the code.  :-)

The Approach:

The hooks code that we currently have is very good, but not quite right
for filters because it is all server-wide in scope.  The first thing I did
was fix that by adding more hooks logic.  We now have RHOOK macros (a one
to one relationship with HOOK macros) which attach a hook to the request
rec.

Moduleys still use ap_r* to output data, but in the apr* function, that
data is copied into a new char (to keep it from being const), and the data
is then passed through all of the filters before it is written out.

It is possible to register a function for a hook at anytime while serving
a request.  I have a module that currently inserts a filter during the
fixups phase based on the request issued.  This also means that different
requests can use different filters.  :-)

Future Improvements in this code:

The filter hook should be modified to either take a char ** and int * or
an iovec instead of a char *and int.  This would make it much easier to
add text to the output.

Filters need to be able to say "I'm holding a piece of this data to
process the next time you send me stuff, please wait" And this needs to be
respected by the code.  For example, if mod_include gets half an SSI tag
at the end of text block, it will hold the half of the tag it has, and
return "I have to hold onto this", and when it gets the rest of the tag,
it will return the correct result.  This means we need to be able to save
the text and the current filters out of the way to be restored later.  If
we don't save the current filter list, then somebody could re-write
history.

We need to be able to remove filters from the list.

The hooks logic can probably be combined a bit with the original hooks
logic to be more concise and have less duplication.

I'm sure there are more, but this is a very nice first draft IMHO.

If anybody would like, I can post my newest mod_gargle later.  Assuming I
get no negative feedback, this patch will be applied immediately following
the next alpha.  This provides about three days for people to review the
patch, the alpha is released Friday, and then we commit filtered I/O.  :-)

The PATCH:

Index: src/CHANGES
===================================================================
RCS file: /home/cvs/apache-2.0/src/CHANGES,v
retrieving revision 1.126
diff -u -d -b -w -u -r1.126 CHANGES
--- src/CHANGES	2000/05/30 02:42:32	1.126
+++ src/CHANGES	2000/05/31 15:07:17
@@ -1,4 +1,5 @@
 Changes with Apache 2.0a4
+  *) Add the ability to filter I/O to Apache 2.0. [Ryan Bloom]
 
   *) We now report the correct line number for syntax errors in config
      files.  [Ryan Bloom, Greg Stein, Jeff Trawick]
Index: src/include/ap_hooks.h
===================================================================
RCS file: /home/cvs/apache-2.0/src/include/ap_hooks.h,v
retrieving revision 1.20
diff -u -d -b -w -u -r1.20 ap_hooks.h
--- src/include/ap_hooks.h	2000/05/27 22:53:46	1.20
+++ src/include/ap_hooks.h	2000/05/31 15:07:18
@@ -167,6 +167,116 @@
     return decline; \
     }
 
+/* Set of Hook definitions designed to put the list of functions off a
+ * pointer pre-defined in the request_rec.
+ */
+#define AP_DECLARE_RHOOK(ret,name,args) \
+typedef ret HOOK_##name args; \
+void ap_rhook_##name(HOOK_##name *pf,request_rec *r,const char * const *aszPre,\
+		    const char * const *aszSucc,int nOrder); \
+ret ap_run_##name args; \
+typedef struct _LINK_##name \
+    { \
+    HOOK_##name *pFunc; \
+    const char *szName; \
+    const char * const *aszPredecessors; \
+    const char * const *aszSuccessors; \
+    int nOrder; \
+    } LINK_##name;
+
+#define AP_RHOOK_STRUCT(members) \
+struct { members } _hooks;
+
+#define AP_RHOOK_LINK(name) \
+    ap_array_header_t *link_##name;
+
+#define AP_IMPLEMENT_RHOOK_BASE(name) \
+void ap_rhook_##name(HOOK_##name *pf,request_rec *r,const char * const *aszPre,\
+		    const char * const *aszSucc,int nOrder) \
+    { \
+    LINK_##name *pHook; \
+    if(!r->_hooks.link_##name) \
+	{ \
+	r->_hooks.link_##name=ap_make_array(r->pool,1,sizeof(LINK_##name)); \
+	ap_hook_sort_register(#name,&r->_hooks.link_##name); \
+	} \
+    pHook=ap_push_array(r->_hooks.link_##name); \
+    pHook->pFunc=pf; \
+    pHook->aszPredecessors=aszPre; \
+    pHook->aszSuccessors=aszSucc; \
+    pHook->nOrder=nOrder; \
+    pHook->szName=ap_debug_module_name; \
+    if(ap_debug_module_hooks) \
+	ap_show_hook(#name,aszPre,aszSucc); \
+    }
+
+/* RUN_ALL runs to the first one to return other than ok or decline
+   RUN_FIRST runs to the first one to return other than decline
+   VOID runs all
+*/
+
+#define AP_IMPLEMENT_RHOOK_VOID(name,args_decl,args_use) \
+AP_IMPLEMENT_RHOOK_BASE(name) \
+void ap_run_##name args_decl \
+    { \
+    LINK_##name *pHook; \
+    int n; \
+\
+    if(!r->_hooks.link_##name) \
+	return; \
+\
+    pHook=(LINK_##name *)r->_hooks.link_##name->elts; \
+    for(n=0 ; n < r->_hooks.link_##name->nelts ; ++n) \
+	pHook[n].pFunc args_use; \
+    }
+
+/* FIXME: note that this returns ok when nothing is run. I suspect it should
+   really return decline, but that breaks Apache currently - Ben
+*/
+#define AP_IMPLEMENT_RHOOK_RUN_ALL(ret,name,args_decl,args_use,ok,decline) \
+AP_IMPLEMENT_RHOOK_BASE(name) \
+ret ap_run_##name args_decl \
+    { \
+    LINK_##name *pHook; \
+    int n; \
+    ret rv; \
+\
+    if(!r->_hooks.link_##name) \
+	return ok; \
+\
+    pHook=(LINK_##name *)r->_hooks.link_##name->elts; \
+    for(n=0 ; n < r->_hooks.link_##name->nelts ; ++n) \
+	{ \
+	rv=pHook[n].pFunc args_use; \
+\
+	if(rv != ok && rv != decline) \
+	    return rv; \
+	} \
+    return ok; \
+    }
+
+#define AP_IMPLEMENT_RHOOK_RUN_FIRST(ret,name,args_decl,args_use,decline) \
+AP_IMPLEMENT_RHOOK_BASE(name) \
+ret ap_run_##name args_decl \
+    { \
+    LINK_##name *pHook; \
+    int n; \
+    ret rv; \
+\
+    if(!r->_hooks.link_##name) \
+	return decline; \
+\
+    pHook=(LINK_##name *)r->_hooks.link_##name->elts; \
+    for(n=0 ; n < r->_hooks.link_##name->nelts ; ++n) \
+	{ \
+	rv=pHook[n].pFunc args_use; \
+\
+	if(rv != decline) \
+	    return rv; \
+	} \
+    return decline; \
+    }
+
      /* Hook orderings */
 #define AP_HOOK_REALLY_FIRST	(-10)
 #define AP_HOOK_FIRST		0
Index: src/include/httpd.h
===================================================================
RCS file: /home/cvs/apache-2.0/src/include/httpd.h,v
retrieving revision 1.49
diff -u -d -b -w -u -r1.49 httpd.h
--- src/include/httpd.h	2000/05/28 17:52:19	1.49
+++ src/include/httpd.h	2000/05/31 15:07:18
@@ -81,6 +81,7 @@
 #include "apr_network_io.h"
 #include "buff.h"
 #include "ap_mmn.h"
+#include "ap_hooks.h"
 
 #ifdef HAVE_NETINET_IN_H
 #include <netinet/in.h>
@@ -606,6 +607,8 @@
     const char *short_name;
 };
 
+AP_DECLARE_RHOOK(void, filter, (request_rec *r, char *str, int len));
+
 struct request_rec {
 
     ap_pool_t *pool;
@@ -622,6 +625,10 @@
     request_rec *main;		/* If this is a sub_request (see request.h) 
 				 * pointer back to the main request.
 				 */
+
+    AP_RHOOK_STRUCT (
+        AP_RHOOK_LINK(filter)   /* per-request filter I/O hook. */
+    );
 
     /* Info about the request itself... we begin with stuff that only
      * protocol.c should ever touch...
Index: src/main/http_protocol.c
===================================================================
RCS file: /home/cvs/apache-2.0/src/main/http_protocol.c,v
retrieving revision 1.74
diff -u -d -b -w -u -r1.74 http_protocol.c
--- src/main/http_protocol.c	2000/05/31 01:35:46	1.74
+++ src/main/http_protocol.c	2000/05/31 15:07:50
@@ -2179,7 +2179,7 @@
         o = 0;
 
         while (n && !ap_is_aborted(r->connection)) {
-            rv = ap_bwrite(r->connection->client, &buf[o], n, &w);
+            rv = ap_rwrite(&buf[o], n, r);
             if (w > 0) {
                 total_bytes_sent += w;
                 n -= w;
@@ -2312,6 +2312,7 @@
     ap_ssize_t w;
     ap_status_t rv;
     char *addr;
+    char *copy;
     
     if (length == 0)
         return 0;
@@ -2328,7 +2329,9 @@
 
         while (n && !r->connection->aborted) {
             ap_mmap_offset((void**)&addr, mm, offset);
-            rv = ap_bwrite(r->connection->client, addr, n, &w);
+            copy = ap_pcalloc(r->pool, n); 
+            memcpy(copy, addr, n); 
+            w = ap_rwrite(copy, n, r);
             if (w > 0) {
                 total_bytes_sent += w;
                 n -= w;
@@ -2360,6 +2363,8 @@
     if (r->connection->aborted)
         return EOF;
 
+    ap_run_filter(r, (char *)&c, 1);
+
     if (ap_bputc(c, r->connection->client) < 0) {
         if (!r->connection->aborted) {
             ap_log_rerror(APLOG_MARK, APLOG_INFO,
@@ -2377,11 +2382,18 @@
 API_EXPORT(int) ap_rputs(const char *str, request_rec *r)
 {
     int rcode;
+    char *newstr;
+    int len;
 
     if (r->connection->aborted)
         return EOF;
     
-    rcode = ap_bputs(str, r->connection->client);
+    len = strlen(str);
+    newstr = ap_pcalloc(r->pool, len + 1);
+    memcpy(newstr, str, len);
+    ap_run_filter(r, newstr, strlen(str));
+
+    rcode = ap_bputs(newstr, r->connection->client);
     if (rcode < 0) {
         if (!r->connection->aborted) {
             ap_log_rerror(APLOG_MARK, APLOG_INFO,
@@ -2400,11 +2412,18 @@
 {
     ap_ssize_t n;
     ap_status_t rv;
+    int len;
+    char *newstr;
 
     if (r->connection->aborted)
         return EOF;
 
-    rv = ap_bwrite(r->connection->client, buf, nbyte, &n);
+    len = strlen(buf);
+    newstr = ap_pcalloc(r->pool, len + 1);
+    memcpy(newstr, buf, len);
+    ap_run_filter(r, newstr, nbyte);
+
+    rv = ap_bwrite(r->connection->client, newstr, nbyte, &n);
     if (n < 0) {
         if (!r->connection->aborted) {
             ap_log_rerror(APLOG_MARK, APLOG_INFO, rv, r,
@@ -2421,11 +2440,13 @@
 API_EXPORT(int) ap_vrprintf(request_rec *r, const char *fmt, va_list ap)
 {
     int n;
+    char *str;
 
     if (r->connection->aborted)
         return -1;
 
-    n = ap_vbprintf(r->connection->client, fmt, ap);
+    str = ap_pvsprintf(r->pool, fmt, ap);    
+    ap_run_filter(r, str, strlen(str));
 
     if (n < 0) {
         if (!r->connection->aborted) {
@@ -2445,14 +2466,18 @@
 {
     va_list vlist;
     int n;
+    char *str;
 
     if (r->connection->aborted)
         return -1;
 
     va_start(vlist, fmt);
-    n = ap_vbprintf(r->connection->client, fmt, vlist);
+    str = ap_pvsprintf(r->pool, fmt, vlist);
     va_end(vlist);
 
+    n = strlen(str);
+    ap_run_filter(r, str, n);
+
     if (n < 0) {
         if (!r->connection->aborted) {
             ap_log_rerror(APLOG_MARK, APLOG_INFO,
@@ -2473,6 +2498,7 @@
     ap_ssize_t i;
     int j, k;
     const char *x;
+    char *str;
     BUFF *fb = r->connection->client;
     ap_status_t rv;
 
@@ -2485,7 +2511,12 @@
         if (x == NULL)
             break;
         j = strlen(x);
-        rv = ap_bwrite(fb, x, j, &i);
+
+        str = ap_pcalloc(r->pool, j + 1); 
+        memcpy(str, x, j); 
+        ap_run_filter(r, str, j);
+
+        rv = ap_bwrite(fb, str, j, &i);
         if (i != j) {
             va_end(args);
             if (!r->connection->aborted) {
@@ -2915,3 +2946,6 @@
                             (const request_rec *r),(r),NULL)
 AP_IMPLEMENT_HOOK_RUN_FIRST(unsigned short,default_port,
                             (const request_rec *r),(r),0)
+AP_IMPLEMENT_RHOOK_VOID(filter, (request_rec *r, char *str, int len),
+                        (r, str, len))
+


Ryan
_______________________________________________________________________________
Ryan Bloom                        	rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------




Mime
View raw message