Received: (from majordom@localhost)
	by hyperreal.org (8.8.5/8.8.5) id IAA26697;
	Tue, 29 Jul 1997 08:52:39 -0700 (PDT)
Received: from colin.muc.de (root@colin.muc.de [193.174.4.1])
	by hyperreal.org (8.8.5/8.8.5) with SMTP id IAA26670
	for <new-httpd@hyperreal.org>; Tue, 29 Jul 1997 08:52:32 -0700 (PDT)
Received: from en by colin.muc.de with UUCP id <86097-2>;
 Tue, 29 Jul 1997 17:52:06 +0200
Received: by en1.engelschall.com (Sendmail 8.8.2) for new-httpd@hyperreal.org
	id RAA11781; Tue, 29 Jul 1997 17:51:06 +0200 (MET DST)
Message-Id: <199707291551.RAA11781@en1.engelschall.com>
Subject: [PATCH] more backref interpolation for rewriting engine
To: new-httpd@hyperreal.org (Apache Developer ML)
Date: Tue, 29 Jul 1997 17:51:05 +0200
From: rse@engelschall.com (Ralf S. Engelschall)
Organization: Engelschall, Germany.
X-Home: http://www.engelschall.com/
X-Mailer: ELM [version 2.4ME+ PL32 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Sender: new-httpd-owner@apache.org
Precedence: bulk
Reply-To: new-httpd@apache.org


While the recent patches to mod_rewrite were pure bugfixes, here is an
essential enhancement patch which I really want to see in 1.3. I'm myself
wondering why this functionality was missing for such a long time, although is
was not difficult to implement and a lot of users wanted it in the past....

Description:
------------

To make it clear, let me explain how a typical ruleset gets evaluated by the
rewriting engine:

   RewriteCond <cond-string-1> <cond-pattern-1>
   RewriteCond <cond-string-2> <cond-pattern-2>
      :
   RewriteCond <cond-string-N> <cond-pattern-N>
   RewriteRule <url-pattern>   <url-string>

The evaluation is this:
   1. Match current URL against <url-pattern>.
      if not match -> ignore rule and all conditions
   2. Create <cond-string-1> and match against <cond-pattern-1>
      if not match -> ignore rule and all conditions
   3. Create <cond-string-2> and match against <cond-pattern-2>
      if not match -> ignore rule and all conditions
        :
 n+1. Create <cond-string-N> and match against <cond-pattern-N>
      if not match -> ignore rule and all conditions
 n+2. Create <url-string> and replace current URL with this one.
 n+3. Proceed with following rules.

The essential part here is that one usually uses regex grouping via
parenthesis in <url-pattern> to match some parts of the URL and then
backreferences $1...$n in <url-string> to use them, i.e. to construct
<url-string> one can use the result of the already matched pattern.

** The restriction: When constructing <cond-string-XX> we cannot use these
backreferences. Neither the one from <url-pattern> nor from and preceeding
<cond-pattern-XX>. And we also cannot use backreferences in <url-string> to
access the results from any successful <cond-pattern-XX>. Two big restrictions
although the information is still there. Just no way to access it in the past.

Now the new functionality: 
   1. One can use $1..$n in any <cond-string-XX> to backreference the result
      of <url-pattern>.
   2. One can use %1..%n in any <cond-string-XX> _AND_ in
      <url-string> to backreference the result from the last matching
      <cond-pattern-XX>. (initially this is empty for <cond-string-1>).

What is the benefit? A lot of rewriting rules can be shortend and a lot of
other are now the first time possible!!

Example: 
-------

Assume we want to generate (e.g. via WML) alternative variants of HTML pages.
We do this by inserting a style tag before the extension, e.g.  index.html is
the HTML page with the standard style while index.min.html is the one
generated for Lynx users. This approach is done for a complete hierarchy of
webpages. 

Now we want to be able to provide these variants to the user in a nice way.
The nicest way is just to be able to insert the style tag at any point in the
URLs of this webarea to switch the variant. For instance /our/area/ should
give the index.html while /our/min/area/ or /min/our/area/ should give the
index.min.html. Same for style variants like "min", "std" and "max", etc. BUT:
When there is no xx.<STYLE>.html then xx.html should be used.  This way we are
backward-compatible and need not to provide a complete clone of the webarea
for each style. 

The trick is that navigation remains homogeneous inside a style variant when
we use only relative links, because the style is always passed on via the URL.

Without the new functionality this is completely impossible!  With this
functionality it is straight-forward:

    #   step 1: always remove the indicator part, but remember it
    RewriteRule   ^(.*)/(min|std|max)/(.*)$  $1/$3  [E=STYLE:$2]
    
    #   step 2: on initial requests check some URLs if they
    #           are valid when we insert the style indicator and
    #           rewrite to this variant when the URLs exists
    RewriteCond   %{ENV:STYLE}              !=""
    RewriteCond   $1.%{ENV:STYLE}.$2        -U
    RewriteRule   ^(.+)\.([sp]?html|s?cgi)$ $1.%{ENV:STYLE}.$2 [NS]

Side effect:
------------

To be able to implement it cleanly I've changed the code of the rewriting
engine to use a common new function expand_backref_inbuffer(). This way the
code shrunks and is really more clear then ever before. Ok, this cannot be
seen directly from the diff snippet below, but have a look at
apply_rewrite_rule() before and after the patch. 

                                       Ralf S. Engelschall
                                       rse@engelschall.com
                                       www.engelschall.com

--- mod_rewrite.h	Tue Jul 29 14:08:53 1997
+++ mod_rewrite.h.NEW	Tue Jul 29 14:08:58 1997
@@ -205,7 +205,8 @@
 
 #define MAX_ENV_FLAGS 5
 
-#define EOS_PARANOIA(ca) ca[sizeof(ca)-1] = '\0'
+#define EOS_PARANOIA(ca)            ca[sizeof(ca)-1] = '\0'
+#define EOS_PARANOIA_SIZE(cp, size) cp[size-1] = '\0'
 
 
 /*
@@ -292,6 +293,15 @@
     array_header *lists;
 } cache;
 
+    /* the regex structure for the
+       substitution of backreferences */
+
+typedef struct backrefinfo {
+    char *source;
+    int nsub;
+    regmatch_t regmatch[10];
+} backrefinfo;
+
 
 /*
 **
@@ -335,11 +345,12 @@
     /* rewriting engine */
 static int apply_rewrite_list(request_rec *r, array_header *rewriterules, char *perdir);
 static int apply_rewrite_rule(request_rec *r, rewriterule_entry *p, char *perdir); 
-static int apply_rewrite_cond(request_rec *r, rewritecond_entry *p, char *perdir); 
+static int apply_rewrite_cond(request_rec *r, rewritecond_entry *p, char *perdir, backrefinfo *briRR, backrefinfo *briRC); 
 
     /* URI transformation function */
 static void  splitout_queryargs(request_rec *r, int qsappend);
 static void  reduce_uri(request_rec *r);
+static void  expand_backref_inbuffer(pool *p, char *buf, int nbuf, backrefinfo *bri, char c);
 static char *expand_tildepaths(request_rec *r, char *uri);
 static void  expand_map_lookups(request_rec *r, char *uri, int uri_len);
 
--- mod_rewrite.c	Tue Jul 29 17:15:40 1997
+++ mod_rewrite.c.NEW	Tue Jul 29 17:47:37 1997
@@ -1442,16 +1442,17 @@
     char newuri[MAX_STRING_LEN];
     char env[MAX_STRING_LEN];
     char port[32];
-    char env2[MAX_STRING_LEN];
     regex_t *regexp;
     regmatch_t regmatch[10];
-    int rc;
+    backrefinfo *briRR = NULL;
+    backrefinfo *briRC = NULL;
     int prefixstrip;
-    int i;
     int failed;
     array_header *rewriteconds;
     rewritecond_entry *conds;
     rewritecond_entry *c;
+    int i;
+    int rc;
 
     uri     = r->filename;
     regexp  = p->regexp;
@@ -1481,6 +1482,24 @@
     if (( rc && !(p->flags & RULEFLAG_NOTMATCH)) ||
         (!rc &&  (p->flags & RULEFLAG_NOTMATCH))   ) {     
 
+        /* create the RewriteRule regsubinfo */
+        briRR = (backrefinfo *)palloc(r->pool, sizeof(backrefinfo));
+        if (!rc && (p->flags & RULEFLAG_NOTMATCH)) {
+            briRR->source = "";
+            briRR->nsub   = 0;
+        }
+        else {
+            briRR->source = pstrdup(r->pool, uri);
+            briRR->nsub   = regexp->re_nsub;
+            memcpy((void *)(briRR->regmatch), (void *)(regmatch), sizeof(regmatch_t)*10);
+        }
+
+        /* create the RewriteCond backrefinfo, but
+           initialized as empty backrefinfo, i.e. not subst */
+        briRC = (backrefinfo *)pcalloc(r->pool, sizeof(backrefinfo));
+        briRC->source = "";
+        briRC->nsub   = 0;
+
         /* ok, the pattern matched, but we now additionally have to check 
            for any preconditions which have to be also true. We do this
            at this very late stage to avoid unnessesary checks which
@@ -1490,7 +1509,7 @@
         failed = 0;
         for (i = 0; i < rewriteconds->nelts; i++) {
             c = &conds[i];
-            rc = apply_rewrite_cond(r, c, perdir);
+            rc = apply_rewrite_cond(r, c, perdir, briRR, briRC);
             if (c->flags & CONDFLAG_ORNEXT) {
                 /* there is a "or" flag */
                 if (rc == 0) {
@@ -1523,10 +1542,10 @@
         if (strcmp(output, "-") == 0) {
             /* but before we set the env variables... */
             for (i = 0; p->env[i] != NULL; i++) {
-                strncpy(env2, p->env[i], sizeof(env2)-1);
-                EOS_PARANOIA(env2);
-                strncpy(env, pregsub(r->pool, env2, uri, regexp->re_nsub+1, regmatch), sizeof(env)-1);    /* substitute in output */
+                strncpy(env, p->env[i], sizeof(env)-1);
                 EOS_PARANOIA(env);
+                expand_backref_inbuffer(r->pool, env, sizeof(env), briRR, '$');
+                expand_backref_inbuffer(r->pool, env, sizeof(env), briRC, '%');
                 add_env_variable(r, env);
             }
             return 2;
@@ -1534,27 +1553,13 @@
 
         /* if this is a forced proxy request ... */
         if (p->flags & RULEFLAG_PROXY) {
-            if (p->flags & RULEFLAG_NOTMATCH) {
-                output = pstrcat(r->pool, "proxy:", output, NULL);
-                strncpy(newuri, output, sizeof(newuri)-1);
-                EOS_PARANOIA(newuri);
-                expand_variables_inbuffer(r, newuri, sizeof(newuri));/* expand %{...} */
-                expand_map_lookups(r, newuri, sizeof(newuri));       /* expand ${...} */
-            }
-            else {
-                output = pstrcat(r->pool, "proxy:", output, NULL);
-                strncpy(newuri, pregsub(r->pool, output, uri, regexp->re_nsub+1, regmatch), sizeof(newuri)-1);    /* substitute in output */
-                EOS_PARANOIA(newuri);
-                for (i = 0; p->env[i] != NULL; i++) {
-                    strncpy(env2, p->env[i], sizeof(env2)-1);
-                    EOS_PARANOIA(env2);
-                    strncpy(env, pregsub(r->pool, env2, uri, regexp->re_nsub+1, regmatch), sizeof(env)-1);    /* substitute in output */
-                    EOS_PARANOIA(env);
-                    add_env_variable(r, env);
-                }
-                expand_variables_inbuffer(r, newuri, sizeof(newuri));   /* expand %{...} */
-                expand_map_lookups(r, newuri, sizeof(newuri));          /* expand ${...} */
-            }
+            output = pstrcat(r->pool, "proxy:", output, NULL);
+            strncpy(newuri, output, sizeof(newuri)-1);
+            EOS_PARANOIA(newuri);
+            expand_backref_inbuffer(r->pool, newuri, sizeof(newuri), briRR, '$'); /* expand $N */
+            expand_backref_inbuffer(r->pool, newuri, sizeof(newuri), briRC, '%'); /* expand %N */
+            expand_variables_inbuffer(r, newuri, sizeof(newuri));                 /* expand %{...} */
+            expand_map_lookups(r, newuri, sizeof(newuri));                        /* expand ${...} */
             if (perdir == NULL)
                 rewritelog(r, 2, "rewrite %s -> %s", r->filename, newuri);
             else
@@ -1570,24 +1575,18 @@
                 || (i > 8 && strncmp(output, "https://", 8) == 0)
                 || (i > 9 && strncmp(output, "gopher://", 9) == 0)
                 || (i > 6 && strncmp(output, "ftp://", 6) == 0)   ) ) {
-            if (p->flags & RULEFLAG_NOTMATCH) {
-                strncpy(newuri, output, sizeof(newuri)-1);
-                EOS_PARANOIA(newuri);
-                expand_variables_inbuffer(r, newuri, sizeof(newuri));/* expand %{...} */
-                expand_map_lookups(r, newuri, sizeof(newuri));       /* expand ${...} */
-            }
-            else {
-                strncpy(newuri, pregsub(r->pool, output, uri, regexp->re_nsub+1, regmatch), sizeof(newuri)-1);    /* substitute in output */
-                EOS_PARANOIA(newuri);
-                for (i = 0; p->env[i] != NULL; i++) {
-                    strncpy(env2, p->env[i], sizeof(env2)-1);
-                    EOS_PARANOIA(env2);
-                    strncpy(env, pregsub(r->pool, env2, uri, regexp->re_nsub+1, regmatch), sizeof(env)-1);    /* substitute in output */
-                    EOS_PARANOIA(env);
-                    add_env_variable(r, env);
-                }
-                expand_variables_inbuffer(r, newuri, sizeof(newuri));/* expand %{...} */
-                expand_map_lookups(r, newuri, sizeof(newuri));       /* expand ${...} */
+            strncpy(newuri, output, sizeof(newuri)-1);
+            EOS_PARANOIA(newuri);
+            expand_backref_inbuffer(r->pool, newuri, sizeof(newuri), briRR, '$'); /* expand $N */
+            expand_backref_inbuffer(r->pool, newuri, sizeof(newuri), briRC, '%'); /* expand %N */
+            expand_variables_inbuffer(r, newuri, sizeof(newuri));                 /* expand %{...} */
+            expand_map_lookups(r, newuri, sizeof(newuri));                        /* expand ${...} */
+            for (i = 0; p->env[i] != NULL; i++) {
+                strncpy(env, p->env[i], sizeof(env)-1);
+                EOS_PARANOIA(env);
+                expand_backref_inbuffer(r->pool, env, sizeof(env), briRR, '$');
+                expand_backref_inbuffer(r->pool, env, sizeof(env), briRC, '%');
+                add_env_variable(r, env);
             }
             rewritelog(r, 2, "[per-dir %s] redirect %s -> %s", perdir, r->filename, newuri);
             r->filename = pstrdup(r->pool, newuri);
@@ -1603,25 +1602,20 @@
             output = pstrcat(r->pool, perdir, output, NULL);
         }
 
-        if (p->flags & RULEFLAG_NOTMATCH) {
-            /* just overtake the URI */
-            strncpy(newuri, output, sizeof(newuri)-1);
-            EOS_PARANOIA(newuri);
-        }
-        else {
-            /* substitute in output */
-            strncpy(newuri, pregsub(r->pool, output, uri, regexp->re_nsub+1, regmatch), sizeof(newuri)-1);    /* substitute in output */
-            EOS_PARANOIA(newuri);
-            for (i = 0; p->env[i] != NULL; i++) {
-                strncpy(env2, p->env[i], sizeof(env2)-1);
-                EOS_PARANOIA(env2);
-                strncpy(env, pregsub(r->pool, env2, uri, regexp->re_nsub+1, regmatch), sizeof(env)-1);    /* substitute in output */
-                EOS_PARANOIA(env);
-                add_env_variable(r, env);
-            }
+        /* standard case: create the substitution string */
+        strncpy(newuri, output, sizeof(newuri)-1);
+        EOS_PARANOIA(newuri);
+        expand_backref_inbuffer(r->pool, newuri, sizeof(newuri), briRR, '$'); /* expand $N */
+        expand_backref_inbuffer(r->pool, newuri, sizeof(newuri), briRC, '%'); /* expand %N */
+        expand_variables_inbuffer(r, newuri, sizeof(newuri));                 /* expand %{...} */
+        expand_map_lookups(r, newuri, sizeof(newuri));                        /* expand ${...} */
+        for (i = 0; p->env[i] != NULL; i++) {
+            strncpy(env, p->env[i], sizeof(env)-1);
+            EOS_PARANOIA(env);
+            expand_backref_inbuffer(r->pool, env, sizeof(env), briRR, '$');
+            expand_backref_inbuffer(r->pool, env, sizeof(env), briRC, '%');
+            add_env_variable(r, env);
         }
-        expand_variables_inbuffer(r, newuri, sizeof(newuri));  /* expand %{...} */
-        expand_map_lookups(r, newuri, sizeof(newuri));         /* expand ${...} */
 
         if (perdir == NULL)
             rewritelog(r, 2, "rewrite %s -> %s", uri, newuri);
@@ -1694,15 +1688,30 @@
     return 0;
 }
 
-static int apply_rewrite_cond(request_rec *r, rewritecond_entry *p, char *perdir)
+static int apply_rewrite_cond(request_rec *r, rewritecond_entry *p, char *perdir, backrefinfo *briRR, backrefinfo *briRC)
 {
-    char *input;
-    int rc;
+    char input[MAX_STRING_LEN];
     struct stat sb;
     request_rec *rsub;
+    regmatch_t regmatch[10];
+    int rc;
 
-    /* first, we have to expand the input string to match */
-    input = expand_variables(r, p->input);
+    /*
+     *   Construct the string we match against
+     */
+
+    /* expand the regex backreferences from the RewriteRule ($0-$9), 
+       then from the last RewriteCond (%0-%9) and then expand the 
+       variables (%{....}) */
+    strncpy(input, p->input, sizeof(input)-1);
+    EOS_PARANOIA(input);
+    expand_backref_inbuffer(r->pool, input, sizeof(input), briRR, '$');
+    expand_backref_inbuffer(r->pool, input, sizeof(input), briRC, '%');
+    expand_variables_inbuffer(r, input, sizeof(input));
+
+    /*
+     *   Apply the patterns
+     */
 
     rc = 0;
     if (strcmp(p->pattern, "-f") == 0) {
@@ -1784,11 +1793,22 @@
         rc = (compare_lexicography(input, p->pattern+1) == -1 ? 1 : 0);
     }
     else if (strlen(p->pattern) > 1 && *(p->pattern) == '=') {
-        rc = (strcmp(input, p->pattern+1) == 0 ? 1 : 0);
+        if (strcmp(p->pattern+1, "\"\"") == 0)
+            rc = (*input == '\0');
+        else
+            rc = (strcmp(input, p->pattern+1) == 0 ? 1 : 0);
     }
     else {
         /* it is really a regexp pattern, so apply it */
-        rc = (regexec(p->regexp, input, 0, NULL, 0) == 0);
+        rc = (regexec(p->regexp, input, p->regexp->re_nsub+1, regmatch, 0) == 0);
+
+        /* if it isn't a negated pattern and really matched
+           we update the passed-through regex subst info structure */
+        if (rc && !(p->flags & CONDFLAG_NOTMATCH)) { 
+            briRC->source = pstrdup(r->pool, input);
+            briRC->nsub   = p->regexp->re_nsub;
+            memcpy((void *)(briRC->regmatch), (void *)(regmatch), sizeof(regmatch_t)*10);
+        }
     }
 
     /* if this is a non-matching regexp, just negate the result */ 
@@ -1934,6 +1954,43 @@
         }
     }
     return;            
+}
+
+
+/*
+**
+**  Expand the %0-%9 or $0-$9 regex backreferences
+**
+*/
+
+static void expand_backref_inbuffer(pool *p, char *buf, int nbuf, backrefinfo *bri, char c)
+{
+    int i;
+
+    if (bri->nsub < 1)
+        return;
+
+    if (c != '$') {
+        /* safe existing $N backrefs and replace <c>N with $N backrefs */
+        for (i = 0; buf[i] != '\0' && i < nbuf; i++) {
+            if (buf[i] == '$' && (buf[i+1] >= '0' && buf[i+1] <= '9'))
+                buf[i++] = '\001';
+            else if (buf[i] == c && (buf[i+1] >= '0' && buf[i+1] <= '9'))
+                buf[i++] = '$';
+        }
+    }
+
+    /* now apply the pregsub() function */
+    strncpy(buf, pregsub(p, buf, bri->source, 
+                         bri->nsub+1, bri->regmatch), nbuf-1);
+    EOS_PARANOIA_SIZE(buf, nbuf);
+
+    if (c != '$') {
+        /* restore the original $N backrefs */
+        for (i = 0; buf[i] != '\0' && i < nbuf; i++)
+            if (buf[i] == '\001' && (buf[i+1] >= '0' && buf[i+1] <= '9'))
+                buf[i++] = '$';
+    }
 }