httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Niklas Edmundsson <>
Subject Re: mod_disk_cache patch, preview edition (was: new cache arch)
Date Tue, 02 May 2006 13:50:40 GMT
On Tue, 2 May 2006, Graham Leggett wrote:

>> The need-size-issue goes for retrievals as well.
> If you are going to read from partially cached files, you need a "total
> size" field as well as a flag to say "give up, this attempt at caching
> failed"

Are there partially cached files? If I request the last 200 bytes of a 
4.3GB DVD image, the bucket brigade contains the complete file... The 
headers says ranges and all sorts of things but they don't match 
what's cached.

> What may be useful is a cache header with some metadata in it giving the
> total size and a "download failed" flag, which goes in front of the
> headers. The metadata can also contain the offset of the body.

I solved it with size in the body and a timeout mechanism, a "download 
failed" flag doesn't cope with segfaults.

>> OK. It's attached. It has only had mild testing using the worker mpm
>> with mmap enabled, it needs a bit more testing and auditing before
>> trusting it too hard.
>> Note that this patch fixes a whole slew of other issues along the way,
>> the most notable ones being LFS on 32bit arch, don't eat all your
>> 32bit memory/address space when caching a huge files, provide
>> r->filename so %f in LogFormat works, and other smaller issues.
> Is it possibly to split the patch into separate fixes for each issue
> (where practical)? It makes it easier to digest.

It's possible, but since I needed to hammer so hard at mod_disk_cache 
to get it in the shape I wanted it I set out to first get the whole 
thing working and then worry about breaking the patch into manageable 
pieces. For example, by doing it all-incremental there would have been 
a dozen or so disk format change-patches, and I really don't think you 
would have wanted that :)

As said, this is a preliminary jumbo patch for those interested in how 
we tackled the various problems involved (or those who love to take 
bleeding edge code for a spin and watch it falling into pieces when 
hitting a weird corner case ;).

> Also the other fixes can be committed immediately/soon, depending on how
> simple they are, which will simplify the final patch.

Yup. I'll update bug#39380 when we feel that we have a good solution.

  Niklas Edmundsson, Admin @ {acc,hpc2n}      |
  To err is Human. To blame someone else is politics.

View raw message