httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Davi Arnaut <>
Subject Re: Possible new cache architecture
Date Tue, 02 May 2006 16:37:39 GMT
On Tue, 2 May 2006 11:22:31 +0200 (MEST)
Niklas Edmundsson <> wrote:

> On Mon, 1 May 2006, Davi Arnaut wrote:
> > More important, if we stick with the key/data concept it's possible to
> > implement the header/body relationship under single or multiple keys.
> I've been hacking on mod_disk_cache to make it:
> * Only store one set of data when one uncached item is accessed
>    simultaneously (currently all requests cache the file and the last
>    finished cache process is "wins").
> * Don't wait until the whole item is cached, reply while caching
>    (currently it stalls).
> * Don't block the requesting thread when requestng a large uncached
>    item, cache in the background and reply while caching (currently it
>    stalls).
> This is mostly aimed at serving huge static files from a slow disk 
> backend (typically an NFS export from a server holding all the disk), 
> such as and .
> Doing this with the current mod_disk_cache disk layout was not 
> possible, doing the above without unneccessary locking means:
> * More or less atomic operations, so caching headers and data in
>    separate files gets very messy if you want to keep consistency.
> * You can't use tempfiles since you want to be able to figure out
>    where the data is to be able to reply while caching.
> * You want to know the size of the data in order to tell when you're
>    done (ie the current size of a file isn't necessarily the real size
>    of the body since it might be caching while we're reading it).
> In the light of our experiences, I really think that you want to have 
> a concept that allows you to keep the bond between header and data. 
> Yes, you can patch up a missing bond by require locking and stuff, but 
> I really prefer not having to lock cache files when doing read access. 
> When it comes to "make the common case fast" a lockless design is very 
> much preferred.

I will repeat once again: there is no locking involved, unless your format
of storing the header/data is really wrong. _The data format is up to
the module using it_, while the storage backend is a completely different

> However, if all those issues are sorted out in the layer above disk 
> cache then the above observations becomes more or less moot.

Yes, that's the point.

> In any case the patch is more or less finished, independent testing 
> and auditing haven't been done yet but I can submit a preliminary 
> jumbo-patch if people are interested in having a look at it now.

Davi Arnaut

View raw message