httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Niklas Edmundsson <>
Subject Re: mod_disk_cache summarization
Date Tue, 24 Oct 2006 10:59:44 GMT
On Mon, 23 Oct 2006, Graham Leggett wrote:

> Was busy cleaning up some other odds and ends, will be back on the cache code 
> again shortly.

I'm awaiting the verdict on how to resolve the "lead request hangs" 
problem before I submit more patches, I feel it's important enough to 
be solved before I start submitting fixes/improvements to the 
following items for mod_disk_cache:

* On disk header fixes to not break when moving between 32/64 bit
   builds, include filename so we can fill in r->filename so %f in
   LogFormat works.
* More assorted small cleanups (mostly error handling).
* Allow disk cache to realise that a (large) file is the same
   regardless of which URL is used to access it. Reduces cache disk
   usage a lot for sites like ours that's known by,,,, and so on.
* Add option to not try to remove cache directories in the cache
   structure. IMHO, this should never be needed since the cache
   directory should not be excessively deep (which the broken defaults
   leads to). Davi had a fix for the cache dir layout I think, and I
   personally think that neither mod_disk_cache nor htcacheclean should
   do rmdir.
* Eventually add option to have header and body in the same cachefile.
* Probably more stuff that I don't remember without looking in the

Also, I suspect that there is documentation that needs to be updated, 
more than just new options.

While working with this I have understood that there are two rather 
different uses for mod_disk_cache: either as a cache in a proxy or as 
a way to make a FTP-server frontend reduce load of its file server 

For the FTP-server frontend usage we see the following 
characteristics: Large files, relatively few requests/s. It's 
important to keep files that are frequently accessed in cache (they 
might be large), hence have cache filesystem mounted with atime and 
clean cache based on atime. This works nicely for us using XFS, and 
cleaning by atime is much quicker and uses less resources than 

Others here are more clued on the proxy-cache-usecase, but as I 
understand it the keywords are many small files, many requests/s so 
need to mount with noatime and use htcacheclean.

Tuning tips in the documentation for these rather different cases 
would probably be apprecieted.

  Niklas Edmundsson, Admin @ {acc,hpc2n}      |
  "All this will be for nothing unless we go to the stars" :  Babylon 5

View raw message