On Tue, 2 May 2006 11:22:31 +0200 (MEST)
Niklas Edmundsson <nikke@acc.umu.se> wrote:
> On Mon, 1 May 2006, Davi Arnaut wrote:
>
> > More important, if we stick with the key/data concept it's possible to
> > implement the header/body relationship under single or multiple keys.
>
> I've been hacking on mod_disk_cache to make it:
> * Only store one set of data when one uncached item is accessed
> simultaneously (currently all requests cache the file and the last
> finished cache process is "wins").
> * Don't wait until the whole item is cached, reply while caching
> (currently it stalls).
> * Don't block the requesting thread when requestng a large uncached
> item, cache in the background and reply while caching (currently it
> stalls).
>
> This is mostly aimed at serving huge static files from a slow disk
> backend (typically an NFS export from a server holding all the disk),
> such as http://ftp.acc.umu.se/ and http://ftp.heanet.ie/ .
>
> Doing this with the current mod_disk_cache disk layout was not
> possible, doing the above without unneccessary locking means:
>
> * More or less atomic operations, so caching headers and data in
> separate files gets very messy if you want to keep consistency.
> * You can't use tempfiles since you want to be able to figure out
> where the data is to be able to reply while caching.
> * You want to know the size of the data in order to tell when you're
> done (ie the current size of a file isn't necessarily the real size
> of the body since it might be caching while we're reading it).
>
> In the light of our experiences, I really think that you want to have
> a concept that allows you to keep the bond between header and data.
> Yes, you can patch up a missing bond by require locking and stuff, but
> I really prefer not having to lock cache files when doing read access.
> When it comes to "make the common case fast" a lockless design is very
> much preferred.
I will repeat once again: there is no locking involved, unless your format
of storing the header/data is really wrong. _The data format is up to
the module using it_, while the storage backend is a completely different
issue.
> However, if all those issues are sorted out in the layer above disk
> cache then the above observations becomes more or less moot.
Yes, that's the point.
> In any case the patch is more or less finished, independent testing
> and auditing haven't been done yet but I can submit a preliminary
> jumbo-patch if people are interested in having a look at it now.
--
Davi Arnaut
|