Mailing-List: contact dev-help@httpd.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@httpd.apache.org
Received-SPF: pass (athena.apache.org: local policy)
User-Agent: Microsoft-Entourage/11.3.6.070618
Date: Mon, 11 Feb 2008 08:38:52 -0500
Subject: Re: cache - cleaning up mod_memcache and making other caches
 their live easier
From: "Akins, Brian" <Brian.Akins@turner.com>
To: "dev@httpd.apache.org" <dev@httpd.apache.org>
Message-ID: <C3D5BA1C.A6AA2%Brian.Akins@turner.com>
Thread-Topic: cache - cleaning up mod_memcache and making other caches
 their live easier
Thread-Index: AchsRE48iCHzS7MGT7uadd5ZUKHtIwAYK9bQAAOcsXo=
In-Reply-To: 
 <99EA83DCDE961346AFA9B5EC33FEC08B464662@VF-MBX11.internal.vodafone.com>
Mime-version: 1.0
Content-type: text/plain;
	charset="US-ASCII"
Content-transfer-encoding: 7bit

If anyone cares, here's how we do keys and vary in our cache:

On store:

Generate key: using url and r->args (we can ignore r->args per server, if
needed) (http://www.domain.com/index.html?you=me)

If(vary) {
    store the following info in meta file:
      cache_version_t - ala disk_cache (ours includes expire time)
      length of key (apr_size_t) including \0
      key with \0
      length of "array" (apr_size_t)
      a \0 delimited array of Vary headers

     regenerate key (basically the original key + vary info:
 http://www.domain.com/index.html?you=meuser-agentmozillaaccept-encodinggzip
)   
}
  
Store key in meta file.  a normal meta file has this format:
    cache_version_t (ours includes expire time)
    length of key (apr_size_t) including \0
    key with \0   
    length of "table" (apr_size_t)
    a \0 delimited table (key\0value\0key\0value\0....) of response headers
    
Note: the reason we use \0 delimited arrays/tables is we read the entire
metafile info into memory on serving and then just apr_table_setn on the
values.  In theory we could mmap the meta files, but we actually found that
to be slower.


On serving:
Generate key: using url and r->args (we can ignore r->args per server, if
needed) (http://www.domain.com/index.html?you=me)

Open metafile

If(vary) {
   thaw vary array
   generate new key (vary values may be overridden by env)
   open new metafile
}

Thaw headers, etc.


So, we only store the headers that we use in vary key calculation.  On my
TODO list is to make key generation a hook bcs we have apps that would
benefit from that.

Of course, we have only one provider (disk) and ignore a lot of RFC stuff
(although we have made most of that configurable), but our key/vary handling
is pretty fast (I spent a lot of time with profilers when writing it).

I'm still working on my side to allow us to actually release the code.

-- 
Brian Akins
Chief Operations Engineer
Turner Digital Media Technologies