httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Niklas Edmundsson <>
Subject Re: mod_cache: store_body() bites off more than it can chew
Date Fri, 03 Sep 2010 14:25:41 GMT
On Thu, 2 Sep 2010, Graham Leggett wrote:

> Should however the cache implementation want to take a breath, it returns to 
> mod_cache with unconsumed bucket(s) still remaining in the "in" brigade. 
> mod_cache in turn sends the already-processed buckets in the "out" brigade 
> down the filter stack to the client, and then loops round, calling the 
> store_body() function again until the "in" brigade is empty.
> In this way, the cache implementation has the option to swallow data in as 
> many smaller chunks as it sees fit, and in turn the client gets fed data 
> often enough to not get bored and time out if the file is very large.
> Regards,
> Graham

This could even go a bit further with providing the cache 
implementation with a hint of when it would be polite of it to return. 
I think it would probably be easier if the cache implementation knows 
what's expected of it. Or?

Also, if the client hangs up, will the cache impl get the chance to 
finish its job (ie. completing the caching of a file instead of 
starting over later on)?

A side-step from this, how would it interact with the thundering herd 
lock and slow client first to access large file while other fast 
clients also wants to access it? Wouldn't this just be another variety 
of the "client gets bored before reply" scenario?

Regarding the issue of the disk cache cramming the entire file into 
memory/address space, an alternate solution could be that the cache 
returns buckets pointing to the cached file, ie that the cache 
consumed those pesky mmapped buckets. This way the cache could cache 
the file rather quickly independent of the speed of the client, so 
that the file caching is finished in a sane time so others can benefit 
from it even though the cache-initiating request is running at 

  Niklas Edmundsson, Admin @ {acc,hpc2n}      |
  "I'm going to hell. It's that simple. I am going straight to hell." - Veronica

View raw message