Mailing-List: contact dev-help@httpd.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@httpd.apache.org
Received-SPF: pass (athena.apache.org: domain of dirkx@webweaving.org
 designates 209.132.96.45 as permitted sender)
Message-Id: <C7568313-3590-4097-9F67-4D193074C7FD@webweaving.org>
From: Dirk-Willem van Gulik <dirkx@webweaving.org>
To: dev@httpd.apache.org
Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes
Content-Transfer-Encoding: 7bit
Mime-Version: 1.0 (Apple Message framework v919.2)
Subject: very rough caching change patch - centralize vary/http knowledge
Date: Wed, 7 May 2008 18:25:38 +0200

Apologies for the size - but about as minimal as I can get it :)

So the problem I see is:

-	current caching modules should understand things such as
	Vary and negotiation. And we're bound to get more.

-	currently only mod_disk_cache does so. There are some 6 or 7
	other modules which ought to get this capability too.

-	we can probably improve the current vary and header
	understanding to get better caching.

Now we could go all out and reduce the mod_disk/mem/distcache/ 
memcached/et.al.
modules to a pure get/set/put API (e.g. see the API's of distcached and
memcached at http://code.sixapart.com/svn/memcached/trunk/server/doc/protocol.txt) 
.

But to me allowing caching modules some knowledge of HTTP is probably  
good - as
optimizing the generic case is simply not worthwhile - the very point  
of caches
is that they understand something of the biz-processes to go beyond  
what a the
operating system and what not can yield.

So my suggestion is to:

-	Fundamentally expect modules to understand Vary.

-	Fundamentally assume that HTTP headers and similar
	caching info is serializable in a few k's to 10's
	of k's.

-	But strip everything out but for the

		get header data for some Key
		deserialize something..

		if (vary_key returned)
			get header data for vary key
			deserialize something
	
		.. and then work on the body
			

-	Move all VARY trickery into cache_util et.al.

As opposed to going the pure key/value get/set/put/del route and
layering something on top of that.

Thoughts - below is some very rough yet functioningish code.

Dw.

Index: mod_cache.h
===================================================================
--- mod_cache.h	(revision 651547)
+++ mod_cache.h	(working copy)
@@ -311,6 +311,74 @@
                                                          apr_table_t  
*t,
                                                          server_rec  
*s);

+/* Serialize a table into a bucked brigade. Assumes that an already
+ * created bucked brigade. Passing an empty table (i.e. null elements)
+ * or a null pointer is treated equally (and the deserialization
+ * will not notice the difference). The table is assumed to contain
+ * textual key and value pairs ('\0' are used as termination tokens).
+ */
+CACHE_DECLARE(apr_status_t)
+ap_brigade_puttable(apr_bucket_brigade * bb, apr_table_t * table);
+
+/* Serialize a table into a bucked brigade. Assumes that an already
+ * created bucked brigade. Passing an empty array (i.e. null elements)
+ * or a null pointer is treated equally (and the deserialization
+ * will not notice the difference). The array is assumed to contain
+ * textual value's ('\0' is used as termination tokens).
+ */
+CACHE_DECLARE(apr_status_t)
+ap_brigade_putarray(apr_bucket_brigade * bb, apr_array_header_t * a);
+
+/* Deserialize a char buffer into an table. Or return a table with
+ * zero element when empty (rather than a null pointer). A table is
+ * created when none is passed; any key value pairs are 'add'-ed
+ * to (any existing) table.
+ */
+CACHE_DECLARE(apr_size_t)
+ap_deserialize_table(apr_pool_t * pool, apr_table_t ** tablep, char  
*in, apr_size_t at, apr_size_t len);
+
+/* Deserialize a char buffer into an array. Or return an arrya with
+ * zero elements when empty (rather than a null pointer). An array is
+ * created when none is passed; any value pairs are 'push'-ed to
+ * the end of (any existing) table.
+ */
+CACHE_DECLARE(apr_size_t)
+ap_deserialize_array(apr_pool_t * pool, apr_array_header_t ** arr,  
char *in, apr_size_t at, apr_size_t len);
+
+CACHE_DECLARE(apr_status_t)ap_serialize_cache_object(
+        apr_pool_t * pool, apr_bucket_alloc_t * bucket_alloc,
+        const char * key,
+
+        apr_bucket_brigade ** serialized_bb,
+
+        const char ** vary_key,
+        apr_bucket_brigade ** serialized_vary_bb,
+
+        cache_info *info,
+
+        apr_table_t * headers_in,
+        apr_table_t * headers_out,
+
+        void * private_block,
+        apr_size_t private_len
+);
+CACHE_DECLARE(apr_status_t)ap_deserialize_cache_object(
+        apr_pool_t * pool,
+        const char * key,
+        const char ** vary_key,
+
+        char * buffp, apr_size_t len,
+
+        char ** urip,
+
+        cache_info * info,
+        apr_table_t ** headers_inp,
+        apr_table_t ** headers_outp,
+
+        void ** private_blockp,
+        apr_size_t * private_lenp
+);
+
  /**
   * cache_storage.c
   */
Index: cache_util.c
===================================================================
--- cache_util.c	(revision 651547)
+++ cache_util.c	(working copy)
@@ -20,6 +20,42 @@

  /* -------------------------------------------------------------- */

+#ifndef AP_CACHE_SERIAL_VERSION
+#define AP_CACHE_SERIAL_VERSION (1)
+#endif
+
+/* Trick the version a little bit - so we're likely to detect little/ 
big
+ * endian mistakes resonably early.
+ */
+#define _FORMAT ((AP_CACHE_SERIAL_VERSION) <<16)
+typedef enum {
+	AP_CACHE_SERIALIZE_UNST_FORMAT_VERSION = _FORMAT + 0,
+	AP_CACHE_SERIALIZE_VARY_FORMAT_VERSION = _FORMAT + 1,
+	AP_CACHE_SERIALIZE_FULL_FORMAT_VERSION = _FORMAT + 2
+} ap_cache_serialize_format_t;
+
+/* Note that this structure is sereialized onto disk - so 32 and 64
+ * byte boundaries should be preserved (generally within httpd
+ * *sizeof(unsigned long) is considered safe ???)
+ */
+typedef struct {
+    /* The size of the entity name that follows. */
+    apr_size_t name_len; /* including terminating \0 */
+
+    /* The number of times we've cached this entity. */
+    apr_size_t entity_version;
+
+    /* Miscellaneous time values. */
+    apr_time_t date;
+    apr_time_t expire;
+    apr_time_t request_time;
+    apr_time_t response_time;
+
+    /* The HTTP status code returned for this response.  */
+    int status;
+} serialized_cache_info_t;
+
+
  extern module AP_MODULE_DECLARE_DATA cache_module;

  /* Determine if "url" matches the hostname, scheme and port and path
@@ -642,6 +678,9 @@
   */
  CACHE_DECLARE(apr_table_t  
*)ap_cache_cacheable_headers_in(request_rec *r)
  {
+    /* XXX to do -- study cache_select -- and further cleans this for
+     *     the 'in' specific case (e.g. Range, If-Modified et.al.
+     */
      return ap_cache_cacheable_headers(r->pool, r->headers_in, r- 
 >server);
  }

@@ -670,3 +709,357 @@

      return headers_out;
  }
+
+/* Serialize a table in a simple '\0' terminated key, value pairs,  
with an
+ * empty key signaling the end. If an empty table is passed (as a  
NULL or
+ * a table with 0 elements) - then write something deserializable as an
+ * empty table.
+ */
+CACHE_DECLARE(apr_status_t)
+ap_brigade_puttable(apr_bucket_brigade * bb, apr_table_t * table)
+{
+        apr_table_entry_t *elts;
+        apr_status_t    rv;
+        int             i;
+
+        if (table) {
+                elts = (apr_table_entry_t *) apr_table_elts(table)- 
 >elts;
+
+                for (i = 0; i < apr_table_elts(table)->nelts; ++i) {
+                        rv = apr_brigade_puts(bb, NULL, NULL,  
elts[i].key);
+                        if (rv != APR_SUCCESS)
+                                return rv;
+
+                        rv = apr_brigade_putc(bb, NULL, NULL, '\0');
+                        if (rv != APR_SUCCESS)
+                                return rv;
+
+                        rv = apr_brigade_puts(bb, NULL, NULL,  
elts[i].val);
+                        if (rv != APR_SUCCESS)
+                                return rv;
+
+                        rv = apr_brigade_putc(bb, NULL, NULL, '\0');
+                        if (rv != APR_SUCCESS)
+                                return rv;
+                };
+        };
+
+        /* and an empty key signals the end. */
+        return apr_brigade_putc(bb, NULL, NULL, '\0');
+}
+
+CACHE_DECLARE(apr_status_t)
+ap_brigade_putarray(apr_bucket_brigade * bb, apr_array_header_t * arr)
+{
+    int i;
+    apr_status_t rv;
+    const char **elts;
+
+    elts = (const char **) arr->elts;
+
+    for (i = 0; i < arr->nelts; i++) {
+	rv = apr_brigade_puts(bb, NULL, NULL, elts[i]);
+        if (rv != APR_SUCCESS)
+            return rv;
+    }
+
+        /* and an empty key signals the end. */
+        return apr_brigade_putc(bb, NULL, NULL, '\0');
+}
+
+#if APR_CHARSET_EBCDIC___XXXX_TO_DO_SOMEWHERE_____
+        /* Chances are that we received an ASCII header text instead of
+         * the expected EBCDIC header lines. Try to auto-detect:
+         */
+        if (!(l = strchr(w, ':'))) {
+            int maybeASCII = 0, maybeEBCDIC = 0;
+            unsigned char *cp, native;
+            apr_size_t inbytes_left, outbytes_left;
+
+            for (cp = w; *cp != '\0'; ++cp) {
+                native = apr_xlate_conv_byte(ap_hdrs_from_ascii, *cp);
+                if (apr_isprint(*cp) && !apr_isprint(native))
+                    ++maybeEBCDIC;
+                if (!apr_isprint(*cp) && apr_isprint(native))
+                    ++maybeASCII;
+            }
+            if (maybeASCII > maybeEBCDIC) {
+                ap_log_error(APLOG_MARK, APLOG_ERR, 0, r->server,
+                             "CGI Interface Error: Script headers  
apparently ASCII: (CGI = %s)",
+                             r->filename);
+                inbytes_left = outbytes_left = cp - w;
+                apr_xlate_conv_buffer(ap_hdrs_from_ascii,
+                                      w, &inbytes_left, w,  
&outbytes_left);
+            }
+        }
+#endif /*APR_CHARSET_EBCDIC*/
+
+/* Note that an empty table is returned as such (i.e. we pass a
+ * table with zero elements) - while it may have been written
+ * with a null pointer.
+ */
+CACHE_DECLARE(apr_size_t)
+ap_deserialize_table(apr_pool_t * pool, apr_table_t ** tablep, char  
*in, apr_size_t at, apr_size_t len)
+{
+        int             j = at;
+
+        if (*tablep == NULL)
+                *tablep = apr_table_make(pool, 20);
+
+        while ((j < len) && in[j]) {
+                char           *key = in + j;
+                j += strlen(key) + 1;
+                char           *val = in + j;
+                j += strlen(val) + 1;
+                apr_table_add(*tablep, key, val);
+        }
+        return j + 1;
+}
+/* Note that an empty arrayis returned as such (i.e. we pass a
+ * table with zero elements) - while it may have been written
+ * with a null pointer.
+ */
+CACHE_DECLARE(apr_size_t)
+ap_deserialize_array(apr_pool_t * pool, apr_array_header_t ** arr,  
char *in, apr_size_t at, apr_size_t len)
+{
+        int             j = at;
+
+        if (*arr== NULL)
+                * arr = apr_array_make(pool, 5, sizeof(char*));
+
+        while ((j < len) && in[j]) {
+                char           *val = in + j;
+                j += strlen(val) + 1;
+                *((const char **)apr_array_push(*arr)) =  
apr_pstrdup(pool, val);
+        }
+        return j + 1;
+}
+
+static int varray_alphasort(const void *fn1, const void *fn2)
+{
+    return strcmp(*(char**)fn1, *(char**)fn2);
+}
+
+static void vary_tokens_to_array(apr_pool_t *p, const char *data,
+                            apr_array_header_t *arr)
+{
+    char *token;
+
+    while ((token = ap_get_list_item(p, &data)) != NULL) {
+        *((const char **) apr_array_push(arr)) = token;
+    }
+
+    /* Sort it so that "Vary: A, B" and "Vary: B, A" are stored the  
same. */
+    qsort((void *) arr->elts, arr->nelts,
+         sizeof(char *), varray_alphasort);
+}
+
+static const char* gen_vary_key(apr_pool_t *p, apr_table_t *headers,
+                             apr_array_header_t *sorted_varray, const  
char *oldkey)
+{
+    struct iovec *iov;
+    unsigned int i, k;
+    int nvec;
+    const char *header;
+    const char **elts;
+
+    nvec = (sorted_varray->nelts * 2) + 1;
+    iov = apr_palloc(p, sizeof(struct iovec) * nvec);
+    elts = (const char **) sorted_varray->elts;
+
+    /* TODO:
+     *    - Handle multiple-value headers better. (sort them?)
+     *    - Handle Case in-sensitive Values better.
+     *        This isn't the end of the world, since it just lowers  
the cache
+     *        hit rate, but it would be nice to fix.
+     *
+     * The majority are case insenstive if they are values (encoding  
etc).
+     * Most of rfc2616 is case insensitive on header contents.
+     *
+     * So the better solution may be to identify headers which should  
be
+     * treated case-sensitive?
+     *  HTTP URI's (3.2.3) [host and scheme are insensitive]
+     *  HTTP method (5.1.1)
+     *  HTTP-date values (3.3.1)
+     *  3.7 Media Types [exerpt]
+     *     The type, subtype, and parameter attribute names are case-
+     *     insensitive. Parameter values might or might not be case- 
sensitive,
+     *     depending on the semantics of the parameter name.
+     *  4.20 Except [exerpt]
+     *     Comparison of expectation values is case-insensitive for  
unquoted
+     *     tokens (including the 100-continue token), and is case- 
sensitive for
+     *     quoted-string expectation-extensions.
+     */
+
+    for(i=0, k=0; i < sorted_varray->nelts; i++) {
+        header = apr_table_get(headers, elts[i]);
+        if (!header) {
+            header = "";
+        }
+        iov[k].iov_base = (char*) elts[i];
+        iov[k].iov_len = strlen(elts[i]);
+        k++;
+        iov[k].iov_base = (char*) header;
+        iov[k].iov_len = strlen(header);
+        k++;
+    }
+    iov[k].iov_base = (char*) oldkey;
+    iov[k].iov_len = strlen(oldkey);
+    k++;
+
+    return apr_pstrcatv(p, iov, k, NULL);
+}
+
+CACHE_DECLARE(apr_status_t)ap_serialize_cache_object(
+        apr_pool_t * pool, apr_bucket_alloc_t * bucket_alloc,
+        const char * key,
+
+        apr_bucket_brigade ** serialized_bb,
+
+        const char ** vary_key,
+        apr_bucket_brigade ** serialized_vary_bb,
+
+        cache_info *info,
+
+        apr_table_t * headers_in,
+        apr_table_t * headers_out,
+
+        void * private_block,
+        apr_size_t private_len
+)
+{
+    const char * vary;
+    apr_uint32_t fmt;
+    apr_status_t rv;
+    serialized_cache_info_t serialized_cache_info;
+
+    *vary_key = NULL;
+    if (headers_out && (vary = apr_table_get(headers_out, "Vary"))) {
+	apr_array_header_t * varray;
+
+	varray = apr_array_make(pool, 6, sizeof(char*));
+        vary_tokens_to_array(pool, vary, varray);
+
+	if (*serialized_vary_bb == NULL)
+		*serialized_vary_bb = apr_brigade_create(pool,bucket_alloc);
+
+	/* XXX package in writev/iovec */
+	fmt = AP_CACHE_SERIALIZE_VARY_FORMAT_VERSION;
+	apr_brigade_write(*serialized_vary_bb, NULL, NULL, (const char  
*)&fmt, sizeof(fmt));
+	apr_brigade_write(*serialized_vary_bb, NULL, NULL, (const char  
*)&info->expire, sizeof(info->expire));
+	ap_brigade_putarray(*serialized_vary_bb, varray);
+
+	*vary_key = gen_vary_key(pool, headers_in, varray, key);
+    };
+
+    serialized_cache_info.date = info->date;
+    serialized_cache_info.expire = info->expire;
+    serialized_cache_info.request_time = info->request_time;
+    serialized_cache_info.response_time = info->response_time;
+    serialized_cache_info.status = info->status;
+    serialized_cache_info.name_len = strlen(key);
+
+    if (*serialized_bb == NULL) {
+	*serialized_bb = apr_brigade_create(pool,bucket_alloc);
+    }
+
+    fmt = AP_CACHE_SERIALIZE_FULL_FORMAT_VERSION;
+    apr_brigade_write(*serialized_bb, NULL, NULL, (const char *)&fmt,  
sizeof(fmt));
+    apr_brigade_write(*serialized_bb, NULL, NULL, (const char  
*)&serialized_cache_info, sizeof(serialized_cache_info));
+    apr_brigade_puts(*serialized_bb, NULL, NULL, key);
+
+    /* Note: always write an (empty) table - even when there are no  
headers,
+     * as we need to de-serialize as well.
+     */
+    rv = ap_brigade_puttable(*serialized_bb, headers_out);
+    if (rv != APR_SUCCESS)
+	return rv;
+
+    rv = ap_brigade_puttable(*serialized_bb, headers_in);
+    if (rv != APR_SUCCESS)
+	return rv;
+
+    if (private_len)
+	    apr_brigade_write(*serialized_bb, NULL, NULL, (const char *)  
private_block, private_len);
+
+    return APR_SUCCESS;
+}
+
+CACHE_DECLARE(apr_status_t)ap_deserialize_cache_object(
+	apr_pool_t * pool,
+	const char * key,
+	const char ** vary_key,
+
+	char * buffp, apr_size_t len,
+
+	char ** urip,
+
+	cache_info * info,
+	apr_table_t ** headers_inp,
+	apr_table_t ** headers_outp,
+
+	void ** private_blockp,
+	apr_size_t * private_lenp
+)
+{
+	serialized_cache_info_t * scip;
+	apr_uint32_t fmt;
+	unsigned int i = 0;
+
+	if (len < sizeof(serialized_cache_info_t))
+		return APR_INCOMPLETE;
+
+	fmt = *(apr_uint32_t *)buffp; i += sizeof(apr_uint32_t);
+
+	/* VARY or FULL header format.. */
+	if (fmt == AP_CACHE_SERIALIZE_VARY_FORMAT_VERSION) {
+		apr_array_header_t * varray = NULL;
+
+		if (vary_key == NULL)
+			return APR_BADARG;
+
+		info->expire = *(apr_time_t *) buffp; i += sizeof(apr_time_t);
+		i = ap_deserialize_array(pool, &varray, buffp, i, len-i);
+
+		*vary_key = gen_vary_key(pool, *headers_inp, varray, key);
+
+		/* return early - and rely on the callee to call
+		 * us again with the 'real' cache data.
+		 */
+		return APR_SUCCESS;
+	} else
+	if (fmt != AP_CACHE_SERIALIZE_FULL_FORMAT_VERSION)
+		return APR_EINVAL;
+
+	/* rely on byte alignment -- i.e. fmt (apr_uint32_t) */
+	scip = (serialized_cache_info_t *) (buffp + i); i +=  
sizeof(serialized_cache_info_t);
+
+        info->date = scip->date;
+        info->expire = scip->expire;
+        info->request_time = scip->request_time;
+        info->response_time = scip->response_time;
+        info->status = scip->status;
+
+        if (len-i < scip->name_len)
+		return APR_EOF;
+
+	*urip = apr_pstrndup(pool,buffp + i,scip->name_len);
+	i += scip->name_len;
+
+	i = ap_deserialize_table(pool, headers_outp, buffp, i, len-i);
+	i = ap_deserialize_table(pool, headers_inp, buffp, i, len-i);
+
+	if (len < i) {
+		if (private_lenp)
+			*private_lenp = len - i;
+		if (private_blockp) {
+			apr_size_t l = len - i;
+			if (private_lenp && *private_lenp < l)
+				l = *private_lenp;
+			if (private_blockp == NULL)
+				*private_blockp = apr_palloc(pool, l);
+			memcpy(*private_blockp, buffp+i, l);
+		};
+	};
+     return APR_SUCCESS;
+}
Index: mod_disk_cache.c
===================================================================
--- mod_disk_cache.c	(revision 651547)
+++ mod_disk_cache.c	(working copy)
@@ -45,7 +45,6 @@
   * Format #2:
   *   disk_cache_info_t (first sizeof(apr_uint32_t) bytes is the  
format)
   *   entity name (dobj->name) [length is in disk_cache_info_t- 
 >name_len]
- *   r->headers_out (delimited by CRLF)
   *   CRLF
   *   r->headers_in (delimited by CRLF)
   *   CRLF
@@ -59,8 +58,6 @@
  static apr_status_t store_body(cache_handle_t *h, request_rec *r,  
apr_bucket_brigade *b);
  static apr_status_t recall_headers(cache_handle_t *h, request_rec *r);
  static apr_status_t recall_body(cache_handle_t *h, apr_pool_t *p,  
apr_bucket_brigade *bb);
-static apr_status_t read_array(request_rec *r, apr_array_header_t* arr,
-                               apr_file_t *file);

  /*
   * Local static functions
@@ -124,35 +121,7 @@
      return APR_SUCCESS;
  }

-/* htcacheclean may remove directories underneath us.
- * So, we'll try renaming three times at a cost of 0.002 seconds.
- */
-static apr_status_t safe_file_rename(disk_cache_conf *conf,
-                                     const char *src, const char *dest,
-                                     apr_pool_t *pool)
-{
-    apr_status_t rv;

-    rv = apr_file_rename(src, dest, pool);
-
-    if (rv != APR_SUCCESS) {
-        int i;
-
-        for (i = 0; i < 2 && rv != APR_SUCCESS; i++) {
-            /* 1000 micro-seconds aka 0.001 seconds. */
-            apr_sleep(1000);
-
-            rv = mkdir_structure(conf, dest, pool);
-            if (rv != APR_SUCCESS)
-                continue;
-
-            rv = apr_file_rename(src, dest, pool);
-        }
-    }
-
-    return rv;
-}
-
  static apr_status_t file_cache_el_final(disk_cache_object_t *dobj,
                                          request_rec *r)
  {
@@ -197,128 +166,6 @@
  }


-/* These two functions get and put state information into the data
- * file for an ap_cache_el, this state information will be read
- * and written transparent to clients of this module
- */
-static int file_cache_recall_mydata(apr_file_t *fd, cache_info *info,
-                                    disk_cache_object_t *dobj,  
request_rec *r)
-{
-    apr_status_t rv;
-    char *urlbuff;
-    disk_cache_info_t disk_info;
-    apr_size_t len;
-
-    /* read the data from the cache file */
-    len = sizeof(disk_cache_info_t);
-    rv = apr_file_read_full(fd, &disk_info, len, &len);
-    if (rv != APR_SUCCESS) {
-        return rv;
-    }
-
-    /* Store it away so we can get it later. */
-    dobj->disk_info = disk_info;
-
-    info->status = disk_info.status;
-    info->date = disk_info.date;
-    info->expire = disk_info.expire;
-    info->request_time = disk_info.request_time;
-    info->response_time = disk_info.response_time;
-
-    /* Note that we could optimize this by conditionally doing the  
palloc
-     * depending upon the size. */
-    urlbuff = apr_palloc(r->pool, disk_info.name_len + 1);
-    len = disk_info.name_len;
-    rv = apr_file_read_full(fd, urlbuff, len, &len);
-    if (rv != APR_SUCCESS) {
-        return rv;
-    }
-    urlbuff[disk_info.name_len] = '\0';
-
-    /* check that we have the same URL */
-    /* Would strncmp be correct? */
-    if (strcmp(urlbuff, dobj->name) != 0) {
-        return APR_EGENERAL;
-    }
-
-    return APR_SUCCESS;
-}
-
-static const char* regen_key(apr_pool_t *p, apr_table_t *headers,
-                             apr_array_header_t *varray, const char  
*oldkey)
-{
-    struct iovec *iov;
-    int i, k;
-    int nvec;
-    const char *header;
-    const char **elts;
-
-    nvec = (varray->nelts * 2) + 1;
-    iov = apr_palloc(p, sizeof(struct iovec) * nvec);
-    elts = (const char **) varray->elts;
-
-    /* TODO:
-     *    - Handle multiple-value headers better. (sort them?)
-     *    - Handle Case in-sensitive Values better.
-     *        This isn't the end of the world, since it just lowers  
the cache
-     *        hit rate, but it would be nice to fix.
-     *
-     * The majority are case insenstive if they are values (encoding  
etc).
-     * Most of rfc2616 is case insensitive on header contents.
-     *
-     * So the better solution may be to identify headers which should  
be
-     * treated case-sensitive?
-     *  HTTP URI's (3.2.3) [host and scheme are insensitive]
-     *  HTTP method (5.1.1)
-     *  HTTP-date values (3.3.1)
-     *  3.7 Media Types [exerpt]
-     *     The type, subtype, and parameter attribute names are case-
-     *     insensitive. Parameter values might or might not be case- 
sensitive,
-     *     depending on the semantics of the parameter name.
-     *  4.20 Except [exerpt]
-     *     Comparison of expectation values is case-insensitive for  
unquoted
-     *     tokens (including the 100-continue token), and is case- 
sensitive for
-     *     quoted-string expectation-extensions.
-     */
-
-    for(i=0, k=0; i < varray->nelts; i++) {
-        header = apr_table_get(headers, elts[i]);
-        if (!header) {
-            header = "";
-        }
-        iov[k].iov_base = (char*) elts[i];
-        iov[k].iov_len = strlen(elts[i]);
-        k++;
-        iov[k].iov_base = (char*) header;
-        iov[k].iov_len = strlen(header);
-        k++;
-    }
-    iov[k].iov_base = (char*) oldkey;
-    iov[k].iov_len = strlen(oldkey);
-    k++;
-
-    return apr_pstrcatv(p, iov, k, NULL);
-}
-
-static int array_alphasort(const void *fn1, const void *fn2)
-{
-    return strcmp(*(char**)fn1, *(char**)fn2);
-}
-
-static void tokens_to_array(apr_pool_t *p, const char *data,
-                            apr_array_header_t *arr)
-{
-    char *token;
-
-    while ((token = ap_get_list_item(p, &data)) != NULL) {
-        *((const char **) apr_array_push(arr)) = token;
-    }
-
-    /* Sort it so that "Vary: A, B" and "Vary: B, A" are stored the  
same. */
-    qsort((void *) arr->elts, arr->nelts,
-         sizeof(char *), array_alphasort);
-}
-
  /*
   * Hook and mod_cache callback functions
   */
@@ -369,19 +216,24 @@

  static int open_entity(cache_handle_t *h, request_rec *r, const char  
*key)
  {
-    apr_uint32_t format;
      apr_size_t len;
      const char *nkey;
      apr_status_t rc;
      static int error_logged = 0;
      disk_cache_conf *conf = ap_get_module_config(r->server- 
 >module_config,
                                                   &disk_cache_module);
-    apr_finfo_t finfo;
      cache_object_t *obj;
      cache_info *info;
      disk_cache_object_t *dobj;
+    unsigned int entity_version;
+    apr_size_t entity_len = sizeof(entity_version);
+    apr_file_t * fd;
+    const char * vary_key;
      int flags;
+    char * uri;

+    char buff[ 1024 * 32 ];	/* XXX replace by hint from cache.h */
+
      h->cache_obj = NULL;

      /* Look up entity keyed to 'url' */
@@ -400,7 +252,6 @@

      info = &(obj->info);

-    /* Open the headers file */
      dobj->prefix = NULL;

      /* Save the cache root */
@@ -408,65 +259,73 @@
      dobj->root_len = conf->cache_root_len;

      dobj->hdrsfile = header_file(r->pool, conf, dobj, key);
+
      flags = APR_READ|APR_BINARY|APR_BUFFERED;
-    rc = apr_file_open(&dobj->hfd, dobj->hdrsfile, flags, 0, r->pool);
+    rc = apr_file_open(&fd, dobj->hdrsfile, flags, 0, r->pool);
      if (rc != APR_SUCCESS) {
-        return DECLINED;
+        ap_log_error(APLOG_MARK, APLOG_DEBUG, 0, r->server,
+            "disk_cache: no header file (%s) found - declining", dobj- 
 >hdrsfile);
+	return DECLINED;
      }

-    /* read the format from the cache file */
-    len = sizeof(format);
-    apr_file_read_full(dobj->hfd, &format, len, &len);
+    len = sizeof(buff);
+    apr_file_read_full(fd, buff, len, &len);
+    apr_file_close(fd);

-    if (format == VARY_FORMAT_VERSION) {
-        apr_array_header_t* varray;
-        apr_time_t expire;
+    vary_key = NULL; uri = NULL;

-        len = sizeof(expire);
-        apr_file_read_full(dobj->hfd, &expire, len, &len);
+    h->resp_hdrs = h->req_hdrs = NULL;

-        varray = apr_array_make(r->pool, 5, sizeof(char*));
-        rc = read_array(r, varray, dobj->hfd);
-        if (rc != APR_SUCCESS) {
-            ap_log_error(APLOG_MARK, APLOG_ERR, rc, r->server,
-                         "disk_cache: Cannot parse vary header file:  
%s",
-                         dobj->hdrsfile);
-            return DECLINED;
-        }
-        apr_file_close(dobj->hfd);
-
-        nkey = regen_key(r->pool, r->headers_in, varray, key);
-
+    rc = ap_deserialize_cache_object(r->pool,
+		key, &vary_key,
+		buff, len,
+		&uri, info,
+		&(h->req_hdrs), &(h->resp_hdrs), /* Note: filling out the  
cache_handle API directly - is that correct; or copy in  
recall_headers ? */
+		(void *) &entity_version, &entity_len
+	 );
+    if (rc != APR_SUCCESS) {
+        ap_log_error(APLOG_MARK, APLOG_ERR, rc, r->server,
+            "disk_cache: deserialize of header file (%s) failed",  
dobj->hdrsfile);
+        return DECLINED;
+    }
+    if (vary_key) {
          dobj->hashfile = NULL;
          dobj->prefix = dobj->hdrsfile;
-        dobj->hdrsfile = header_file(r->pool, conf, dobj, nkey);
+        dobj->hdrsfile = header_file(r->pool, conf, dobj,  
vary_key); // XX we rely on hierachy in remove_url

          flags = APR_READ|APR_BINARY|APR_BUFFERED;
-        rc = apr_file_open(&dobj->hfd, dobj->hdrsfile, flags, 0, r- 
 >pool);
+        rc = apr_file_open(&fd, dobj->hdrsfile, flags, 0, r->pool);
+        if (rc != APR_SUCCESS) return DECLINED;
+        len = sizeof(buff);
+        apr_file_read_full(fd, buff, len, &len);
+        apr_file_close(fd);
+
+        rc = ap_deserialize_cache_object(r->pool,
+		      vary_key, NULL,
+          	buff, len,
+    		   &uri, info,
+             &(h->req_hdrs), &(h->resp_hdrs), /* Note: filling out  
the cache_handle API directly - is that correct; or copy in  
recall_headers ? */
+	        	(void *) &entity_version, &entity_len
+	     );
          if (rc != APR_SUCCESS) {
+            ap_log_error(APLOG_MARK, APLOG_ERR, rc, r->server,
+               "disk_cache: deserialize of viaried header file (%s)  
failed", dobj->hdrsfile);
              return DECLINED;
-        }
+       }
+       nkey = vary_key;
      }
-    else if (format != DISK_FORMAT_VERSION) {
-        ap_log_error(APLOG_MARK, APLOG_ERR, 0, r->server,
-                     "disk_cache: File '%s' has a version mismatch.  
File had version: %d.",
-                     dobj->hdrsfile, format);
-        return DECLINED;
-    }
      else {
-        apr_off_t offset = 0;
-        /* This wasn't a Vary Format file, so we must seek to the
-         * start of the file again, so that later reads work.
-         */
-        apr_file_seek(dobj->hfd, APR_SET, &offset);
          nkey = key;
      }

+    // if (entity_len != sizeof(entity_version)) .. assert
+
      obj->key = nkey;
      dobj->key = nkey;
      dobj->name = key;
      dobj->datafile = data_file(r->pool, conf, dobj, nkey);
      dobj->tempfile = apr_pstrcat(r->pool, conf->cache_root,  
AP_TEMPFILE, NULL);
+    dobj->entity_version = entity_version;

      /* Open the data file */
      flags = APR_READ|APR_BINARY;
@@ -476,23 +335,10 @@
      rc = apr_file_open(&dobj->fd, dobj->datafile, flags, 0, r->pool);
      if (rc != APR_SUCCESS) {
          ap_log_error(APLOG_MARK, APLOG_ERR, rc, r->server,
-                 "disk_cache: Cannot open info header file %s",  dobj- 
 >datafile);
+                 "disk_cache: Cannot open data file %s",  dobj- 
 >datafile);
          return DECLINED;
      }

-    rc = apr_file_info_get(&finfo, APR_FINFO_SIZE, dobj->fd);
-    if (rc == APR_SUCCESS) {
-        dobj->file_size = finfo.size;
-    }
-
-    /* Read the bytes to setup the cache_info fields */
-    rc = file_cache_recall_mydata(dobj->hfd, info, dobj, r);
-    if (rc != APR_SUCCESS) {
-        ap_log_error(APLOG_MARK, APLOG_ERR, rc, r->server,
-                 "disk_cache: Cannot read header file %s",  dobj- 
 >hdrsfile);
-        return DECLINED;
-    }
-
      /* Initialize the cache_handle callback functions */
      ap_log_error(APLOG_MARK, APLOG_DEBUG, 0, r->server,
                   "disk_cache: Recalled cached URL info header %s",   
dobj->name);
@@ -593,154 +439,6 @@
      return OK;
  }

-static apr_status_t read_array(request_rec *r, apr_array_header_t* arr,
-                               apr_file_t *file)
-{
-    char w[MAX_STRING_LEN];
-    int p;
-    apr_status_t rv;
-
-    while (1) {
-        rv = apr_file_gets(w, MAX_STRING_LEN - 1, file);
-        if (rv != APR_SUCCESS) {
-            ap_log_rerror(APLOG_MARK, APLOG_ERR, 0, r,
-                          "Premature end of vary array.");
-            return rv;
-        }
-
-        p = strlen(w);
-        if (p > 0 && w[p - 1] == '\n') {
-            if (p > 1 && w[p - 2] == CR) {
-                w[p - 2] = '\0';
-            }
-            else {
-                w[p - 1] = '\0';
-            }
-        }
-
-        /* If we've finished reading the array, break out of the  
loop. */
-        if (w[0] == '\0') {
-            break;
-        }
-
-       *((const char **) apr_array_push(arr)) = apr_pstrdup(r->pool,  
w);
-    }
-
-    return APR_SUCCESS;
-}
-
-static apr_status_t store_array(apr_file_t *fd, apr_array_header_t*  
arr)
-{
-    int i;
-    apr_status_t rv;
-    struct iovec iov[2];
-    apr_size_t amt;
-    const char **elts;
-
-    elts = (const char **) arr->elts;
-
-    for (i = 0; i < arr->nelts; i++) {
-        iov[0].iov_base = (char*) elts[i];
-        iov[0].iov_len = strlen(elts[i]);
-        iov[1].iov_base = CRLF;
-        iov[1].iov_len = sizeof(CRLF) - 1;
-
-        rv = apr_file_writev(fd, (const struct iovec *) &iov, 2,
-                             &amt);
-        if (rv != APR_SUCCESS) {
-            return rv;
-        }
-    }
-
-    iov[0].iov_base = CRLF;
-    iov[0].iov_len = sizeof(CRLF) - 1;
-
-    return apr_file_writev(fd, (const struct iovec *) &iov, 1,
-                         &amt);
-}
-
-static apr_status_t read_table(cache_handle_t *handle, request_rec *r,
-                               apr_table_t *table, apr_file_t *file)
-{
-    char w[MAX_STRING_LEN];
-    char *l;
-    int p;
-    apr_status_t rv;
-
-    while (1) {
-
-        /* ### What about APR_EOF? */
-        rv = apr_file_gets(w, MAX_STRING_LEN - 1, file);
-        if (rv != APR_SUCCESS) {
-            ap_log_rerror(APLOG_MARK, APLOG_ERR, 0, r,
-                          "Premature end of cache headers.");
-            return rv;
-        }
-
-        /* Delete terminal (CR?)LF */
-
-        p = strlen(w);
-        /* Indeed, the host's '\n':
-           '\012' for UNIX; '\015' for MacOS; '\025' for OS/390
-           -- whatever the script generates.
-        */
-        if (p > 0 && w[p - 1] == '\n') {
-            if (p > 1 && w[p - 2] == CR) {
-                w[p - 2] = '\0';
-            }
-            else {
-                w[p - 1] = '\0';
-            }
-        }
-
-        /* If we've finished reading the headers, break out of the  
loop. */
-        if (w[0] == '\0') {
-            break;
-        }
-
-#if APR_CHARSET_EBCDIC
-        /* Chances are that we received an ASCII header text instead of
-         * the expected EBCDIC header lines. Try to auto-detect:
-         */
-        if (!(l = strchr(w, ':'))) {
-            int maybeASCII = 0, maybeEBCDIC = 0;
-            unsigned char *cp, native;
-            apr_size_t inbytes_left, outbytes_left;
-
-            for (cp = w; *cp != '\0'; ++cp) {
-                native = apr_xlate_conv_byte(ap_hdrs_from_ascii, *cp);
-                if (apr_isprint(*cp) && !apr_isprint(native))
-                    ++maybeEBCDIC;
-                if (!apr_isprint(*cp) && apr_isprint(native))
-                    ++maybeASCII;
-            }
-            if (maybeASCII > maybeEBCDIC) {
-                ap_log_error(APLOG_MARK, APLOG_ERR, 0, r->server,
-                             "CGI Interface Error: Script headers  
apparently ASCII: (CGI = %s)",
-                             r->filename);
-                inbytes_left = outbytes_left = cp - w;
-                apr_xlate_conv_buffer(ap_hdrs_from_ascii,
-                                      w, &inbytes_left, w,  
&outbytes_left);
-            }
-        }
-#endif /*APR_CHARSET_EBCDIC*/
-
-        /* if we see a bogus header don't ignore it. Shout and scream  
*/
-        if (!(l = strchr(w, ':'))) {
-            return APR_EGENERAL;
-        }
-
-        *l++ = '\0';
-        while (*l && apr_isspace(*l)) {
-            ++l;
-        }
-
-        apr_table_add(table, w, l);
-    }
-
-    return APR_SUCCESS;
-}
-
  /*
   * Reads headers from a buffer and returns an array of headers.
   * Returns NULL on file error
@@ -752,22 +450,9 @@
  {
      disk_cache_object_t *dobj = (disk_cache_object_t *) h->cache_obj- 
 >vobj;

-    /* This case should not happen... */
-    if (!dobj->hfd) {
-        ap_log_error(APLOG_MARK, APLOG_ERR, 0, r->server,
-                 "disk_cache: recalling headers; but no header fd for  
%s", dobj->name);
-        return APR_NOTFOUND;
-    }
-
-    h->req_hdrs = apr_table_make(r->pool, 20);
-    h->resp_hdrs = apr_table_make(r->pool, 20);
-
-    /* Call routine to read the header lines/status line */
-    read_table(h, r, h->resp_hdrs, dobj->hfd);
-    read_table(h, r, h->req_hdrs, dobj->hfd);
-
-    apr_file_close(dobj->hfd);
-
+    /* No work needed - the deserialiation during the open has  
already filled
+     * out the req_hdrs and resp_headers in this particular case.
+     */
      ap_log_error(APLOG_MARK, APLOG_DEBUG, 0, r->server,
                   "disk_cache: Recalled headers for URL %s",  dobj- 
 >name);
      return APR_SUCCESS;
@@ -786,206 +471,150 @@
      return APR_SUCCESS;
  }

-static apr_status_t store_table(apr_file_t *fd, apr_table_t *table)
+/* Note that this is not a generic function which can handle any size  
of
+ * bucked brigade - we're sort of assuming resonable header/struct type
+ * of beasts as created by the serialization.
+ */
+static apr_status_t safe_write_brigade_to_file(
+	disk_cache_conf *conf, const char * file,
+	apr_bucket_brigade *bb, apr_pool_t * pool)
  {
-    int i;
-    apr_status_t rv;
-    struct iovec iov[4];
+    char * tempfile = apr_pstrcat(pool, conf->cache_root,  
AP_TEMPFILE, NULL);
+    struct iovec iov[1024]; int iov_len = 1024;
+    apr_file_t * fd;
      apr_size_t amt;
-    apr_table_entry_t *elts;
+    apr_status_t rv;

-    elts = (apr_table_entry_t *) apr_table_elts(table)->elts;
-    for (i = 0; i < apr_table_elts(table)->nelts; ++i) {
-        if (elts[i].key != NULL) {
-            iov[0].iov_base = elts[i].key;
-            iov[0].iov_len = strlen(elts[i].key);
-            iov[1].iov_base = ": ";
-            iov[1].iov_len = sizeof(": ") - 1;
-            iov[2].iov_base = elts[i].val;
-            iov[2].iov_len = strlen(elts[i].val);
-            iov[3].iov_base = CRLF;
-            iov[3].iov_len = sizeof(CRLF) - 1;
+    /* Next 6 functions are really a concurrency proof writeout of a
+     * brigade->tempfile->atomic-name XXX - add apr call for this ?
+     */
+    tempfile = apr_pstrcat(pool, conf->cache_root, AP_TEMPFILE, NULL);

-            rv = apr_file_writev(fd, (const struct iovec *) &iov, 4,
-                                 &amt);
-            if (rv != APR_SUCCESS) {
-                return rv;
-            }
-        }
-    }
-    iov[0].iov_base = CRLF;
-    iov[0].iov_len = sizeof(CRLF) - 1;
-    rv = apr_file_writev(fd, (const struct iovec *) &iov, 1,
-                         &amt);
-    return rv;
-}
+    rv = apr_file_mktemp(&fd,tempfile,
+                         APR_CREATE | APR_WRITE | APR_BINARY |
+                         APR_BUFFERED | APR_EXCL, pool);

-static apr_status_t store_headers(cache_handle_t *h, request_rec *r,  
cache_info *info)
-{
-    disk_cache_conf *conf = ap_get_module_config(r->server- 
 >module_config,
-                                                 &disk_cache_module);
-    apr_status_t rv;
-    apr_size_t amt;
-    disk_cache_object_t *dobj = (disk_cache_object_t*) h->cache_obj- 
 >vobj;
+    if (rv != APR_SUCCESS)
+        return rv;

-    disk_cache_info_t disk_info;
-    struct iovec iov[2];
+    rv = apr_brigade_to_iovec(bb, iov, &iov_len);
+    if (rv != APR_SUCCESS)
+        return rv;

-    /* This is flaky... we need to manage the cache_info differently */
-    h->cache_obj->info = *info;
+    rv = apr_file_writev(fd, iov, iov_len, &amt);
+    if (rv != APR_SUCCESS)
+        return rv;

-    if (r->headers_out) {
-        const char *tmp;
+    rv = apr_file_close(fd);
+    if (rv != APR_SUCCESS)
+        return rv;

-        tmp = apr_table_get(r->headers_out, "Vary");
+    /* Remove old file with the same name. If remove fails, then
+     * perhaps we need to create the directory tree where we are
+     * about to write the new headers file.
+     */
+    rv = apr_file_remove(file, pool);
+    if (rv != APR_SUCCESS) {
+        rv = mkdir_structure(conf, file, pool);
+    }

-        if (tmp) {
-            apr_array_header_t* varray;
-            apr_uint32_t format = VARY_FORMAT_VERSION;
+    /* htcacheclean may remove directories underneath us.
+     *
+     * So, we'll try renaming three times at a cost of 0.002 seconds.
+     *
+     * XXX several patches in the bugtraker to make this better/safer.
+     */
+    rv = apr_file_rename(tempfile, file, pool);

-            /* If we were initially opened as a vary format, rollback
-             * that internal state for the moment so we can recreate  
the
-             * vary format hints in the appropriate directory.
-             */
-            if (dobj->prefix) {
-                dobj->hdrsfile = dobj->prefix;
-                dobj->prefix = NULL;
-            }
+    if (rv != APR_SUCCESS) {
+        int i;

-            rv = mkdir_structure(conf, dobj->hdrsfile, r->pool);
+        for (i = 0; i < 2 && rv != APR_SUCCESS; i++) {
+            /* 1000 micro-seconds aka 0.001 seconds. */
+            apr_sleep(1000);

-            rv = apr_file_mktemp(&dobj->tfd, dobj->tempfile,
-                                 APR_CREATE | APR_WRITE | APR_BINARY  
| APR_EXCL,
-                                 r->pool);
+            rv = mkdir_structure(conf, file, pool);
+            if (rv != APR_SUCCESS)
+                continue;

-            if (rv != APR_SUCCESS) {
-                ap_log_error(APLOG_MARK, APLOG_WARNING, rv, r->server,
-                    "disk_cache: could not create temp file %s",
-                    dobj->tempfile);
-                return rv;
-            }
-
-            amt = sizeof(format);
-            apr_file_write(dobj->tfd, &format, &amt);
-
-            amt = sizeof(info->expire);
-            apr_file_write(dobj->tfd, &info->expire, &amt);
-
-            varray = apr_array_make(r->pool, 6, sizeof(char*));
-            tokens_to_array(r->pool, tmp, varray);
-
-            store_array(dobj->tfd, varray);
-
-            apr_file_close(dobj->tfd);
-
-            dobj->tfd = NULL;
-
-            rv = safe_file_rename(conf, dobj->tempfile, dobj->hdrsfile,
-                                  r->pool);
-            if (rv != APR_SUCCESS) {
-                ap_log_error(APLOG_MARK, APLOG_WARNING, rv, r->server,
-                    "disk_cache: rename tempfile to varyfile failed:  
%s -> %s",
-                    dobj->tempfile, dobj->hdrsfile);
-                    apr_file_remove(dobj->tempfile, r->pool);
-                return rv;
-            }
-
-            dobj->tempfile = apr_pstrcat(r->pool, conf->cache_root,  
AP_TEMPFILE, NULL);
-            tmp = regen_key(r->pool, r->headers_in, varray, dobj- 
 >name);
-            dobj->prefix = dobj->hdrsfile;
-            dobj->hashfile = NULL;
-            dobj->datafile = data_file(r->pool, conf, dobj, tmp);
-            dobj->hdrsfile = header_file(r->pool, conf, dobj, tmp);
+            rv = apr_file_rename(tempfile, file, pool);
          }
      }
-
-
-    rv = apr_file_mktemp(&dobj->hfd, dobj->tempfile,
-                         APR_CREATE | APR_WRITE | APR_BINARY |
-                         APR_BUFFERED | APR_EXCL, r->pool);
-
      if (rv != APR_SUCCESS) {
-       ap_log_error(APLOG_MARK, APLOG_WARNING, rv, r->server,
-           "disk_cache: could not create temp file %s",
-           dobj->tempfile);
+        apr_file_remove(tempfile, pool);
          return rv;
      }

-    disk_info.format = DISK_FORMAT_VERSION;
-    disk_info.date = info->date;
-    disk_info.expire = info->expire;
-    disk_info.entity_version = dobj->disk_info.entity_version++;
-    disk_info.request_time = info->request_time;
-    disk_info.response_time = info->response_time;
-    disk_info.status = info->status;
+    return rv;
+}

-    disk_info.name_len = strlen(dobj->name);
+static apr_status_t store_headers(cache_handle_t *h, request_rec *r,  
cache_info *info)
+{
+    disk_cache_conf *conf = ap_get_module_config(r->server- 
 >module_config,
+                                                 &disk_cache_module);
+    disk_cache_object_t *dobj = (disk_cache_object_t*) h->cache_obj- 
 >vobj;
+    apr_status_t rv;

-    iov[0].iov_base = (void*)&disk_info;
-    iov[0].iov_len = sizeof(disk_cache_info_t);
-    iov[1].iov_base = (void*)dobj->name;
-    iov[1].iov_len = disk_info.name_len;
+    apr_table_t *headers_out = NULL;
+    apr_table_t *headers_in = NULL;

-    rv = apr_file_writev(dobj->hfd, (const struct iovec *) &iov, 2,  
&amt);
-    if (rv != APR_SUCCESS) {
-       ap_log_error(APLOG_MARK, APLOG_WARNING, rv, r->server,
-           "disk_cache: could not write info to header file %s",
-           dobj->hdrsfile);
-        return rv;
-    }
+    apr_bucket_brigade * bb = NULL, * vary_bb = NULL;
+    const char * varykey = NULL;

-    if (r->headers_out) {
-        apr_table_t *headers_out;
+    /* This is flaky... we need to manage the cache_info differently */
+    h->cache_obj->info = *info;

+    if (r->headers_out)
          headers_out = ap_cache_cacheable_headers_out(r);

-        rv = store_table(dobj->hfd, headers_out);
-        if (rv != APR_SUCCESS) {
-           ap_log_error(APLOG_MARK, APLOG_WARNING, rv, r->server,
-               "disk_cache: could not write out-headers to header  
file %s",
-               dobj->hdrsfile);
-            return rv;
-        }
-    }
+    if (r->headers_in)
+        headers_in = ap_cache_cacheable_headers_in(r);

-    /* Parse the vary header and dump those fields from the  
headers_in. */
-    /* FIXME: Make call to the same thing cache_select calls to crack  
Vary. */
-    if (r->headers_in) {
-        apr_table_t *headers_in;
+    dobj->entity_version ++; /* XXX ask some-one what this is/was XXX  
*/

-        headers_in = ap_cache_cacheable_headers_in(r);
+    rv = ap_serialize_cache_object(
+		r->pool,r->connection->bucket_alloc,
+		dobj->name, &bb,
+		&varykey, &vary_bb,
+		info,
+		headers_in,
+		headers_out,
+		&(dobj->entity_version),
+		sizeof(dobj->entity_version)
+    );	

-        rv = store_table(dobj->hfd, headers_in);
-        if (rv != APR_SUCCESS) {
-           ap_log_error(APLOG_MARK, APLOG_WARNING, rv, r->server,
-               "disk_cache: could not write in-headers to header file  
%s",
+    if (rv != APR_SUCCESS) {
+        ap_log_error(APLOG_MARK, APLOG_WARNING, rv, r->server,
+               "disk_cache: could not serialize header file %s",
                 dobj->hdrsfile);
-            return rv;
-        }
+        return rv;
      }

-    apr_file_close(dobj->hfd); /* flush and close */
+    if (varykey) {
+	/* Write the vary information at the URI-key location. */
+    	rv = safe_write_brigade_to_file(conf, dobj->hdrsfile, bb, r- 
 >pool);
+	if (rv != APR_SUCCESS) {
+		ap_log_error(APLOG_MARK, APLOG_WARNING, rv, r->server,
+               		"disk_cache: could not store vary-header file %s",
+               		dobj->hdrsfile);
+		return rv;
+	}
+	/* XXX - should we delete obj->datafile -- which may be left over  
from a non-vary call ? */

-    /* Remove old file with the same name. If remove fails, then
-     * perhaps we need to create the directory tree where we are
-     * about to write the new headers file.
-     */
-    rv = apr_file_remove(dobj->hdrsfile, r->pool);
-    if (rv != APR_SUCCESS) {
-        rv = mkdir_structure(conf, dobj->hdrsfile, r->pool);
-    }
+	/* And adjust the 'real' header of this request to point to a URI 
+vary key */
+        dobj->prefix = dobj->hdrsfile;
+        dobj->hashfile = NULL;
+        dobj->datafile = data_file(r->pool, conf, dobj, varykey);
+        dobj->hdrsfile = header_file(r->pool, conf, dobj, varykey);
+    };

-    rv = safe_file_rename(conf, dobj->tempfile, dobj->hdrsfile, r- 
 >pool);
+    rv = safe_write_brigade_to_file(conf, dobj->hdrsfile, bb, r->pool);
      if (rv != APR_SUCCESS) {
-        ap_log_error(APLOG_MARK, APLOG_WARNING, rv, r->server,
-                     "disk_cache: rename tempfile to hdrsfile failed:  
%s -> %s",
-                     dobj->tempfile, dobj->hdrsfile);
-        apr_file_remove(dobj->tempfile, r->pool);
+       ap_log_error(APLOG_MARK, APLOG_WARNING, rv, r->server,
+           "disk_cache: could not store %s", dobj->hdrsfile);
          return rv;
      }

-    dobj->tempfile = apr_pstrcat(r->pool, conf->cache_root,  
AP_TEMPFILE, NULL);
-
      ap_log_error(APLOG_MARK, APLOG_DEBUG, 0, r->server,
                   "disk_cache: Stored headers for URL %s",  dobj- 
 >name);
      return APR_SUCCESS;
Index: mod_disk_cache.h
===================================================================
--- mod_disk_cache.h	(revision 651547)
+++ mod_disk_cache.h	(working copy)
@@ -35,22 +35,6 @@
  #define AP_TEMPFILE_NAMELEN strlen(AP_TEMPFILE_BASE  
AP_TEMPFILE_SUFFIX)
  #define AP_TEMPFILE AP_TEMPFILE_PREFIX AP_TEMPFILE_BASE  
AP_TEMPFILE_SUFFIX

-typedef struct {
-    /* Indicates the format of the header struct stored on-disk. */
-    apr_uint32_t format;
-    /* The HTTP status code returned for this response.  */
-    int status;
-    /* The size of the entity name that follows. */
-    apr_size_t name_len;
-    /* The number of times we've cached this entity. */
-    apr_size_t entity_version;
-    /* Miscellaneous time values. */
-    apr_time_t date;
-    apr_time_t expire;
-    apr_time_t request_time;
-    apr_time_t response_time;
-} disk_cache_info_t;
-
  /*
   * disk_cache_object_t
   * Pointed to by cache_object_t::vobj
@@ -69,7 +53,7 @@
      apr_file_t *hfd;         /* headers file */
      apr_file_t *tfd;         /* temporary file for data */
      apr_off_t file_size;     /*  File size of the cached data file  */
-    disk_cache_info_t disk_info; /* Header information. */
+    unsigned int entity_version; /* XXX not clear what this is */
  } disk_cache_object_t;

Index: mod_distcached.cpp
===================================================================
No baseline: mod_distcached.cpp
Index: mod_distcached.hpp
===================================================================
No baseline: mod_distcached.hpp
Index: mod_memcached.c
===================================================================
No baseline: mod_memcached.c
Indexe: mod_memcached.h
===================================================================
No baseline: mod_memcached.h
Index: mod_bdb4_cache.c
===================================================================
No baseline: mod_bdb4_cache.c
Index: mod_distdd_BSD_LIC.c
===================================================================
No baseline: mod_distdd_BSD_LIC.c