httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Graham Leggett" <minf...@sharp.fm>
Subject Re: mod_disk_cache patch, preview edition (was: new cache arch)
Date Tue, 02 May 2006 13:31:51 GMT
On Tue, May 2, 2006 2:03 pm, Niklas Edmundsson said:

>> This is great, in doing this you've been solving a proxy bug that was
>> first reported in 1998 :).
>
> OK. Stuck in the "File under L for Later" pile? ;)

Er no, it was under the "redesign the entire code to fix it" class of
bugs. :)

The v2.0 mod_cache design had provision for solving this problem, but it
was never completed. The v1.3 mod_proxy/cache design needed a major
rewrite to fix, the effort was instead put into v2.0.

> Regarding partially cached files, it understands when caching a file
> has failed and so on.

All the cache has to worry about is invalidating all partially cached
files where an upstream error occurred (timeout, connection reset by peer,
whatever) the end goal being to never inadvertantly cache a broken file.

> They are. It seek():s to an offset where the body is stored so
> headers can be updated as long as they don't grow too much.

Ok, makes sense.

> The need-size-issue goes for retrievals as well.

If you are going to read from partially cached files, you need a "total
size" field as well as a flag to say "give up, this attempt at caching
failed"

What may be useful is a cache header with some metadata in it giving the
total size and a "download failed" flag, which goes in front of the
headers. The metadata can also contain the offset of the body.

> OK. It's attached. It has only had mild testing using the worker mpm
> with mmap enabled, it needs a bit more testing and auditing before
> trusting it too hard.
>
> Note that this patch fixes a whole slew of other issues along the way,
> the most notable ones being LFS on 32bit arch, don't eat all your
> 32bit memory/address space when caching a huge files, provide
> r->filename so %f in LogFormat works, and other smaller issues.

Is it possibly to split the patch into separate fixes for each issue
(where practical)? It makes it easier to digest.

Also the other fixes can be committed immediately/soon, depending on how
simple they are, which will simplify the final patch.

Regards,
Graham
--



Mime
View raw message