httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brian Pane <>
Subject Re: [PATCH] mod_cache: support caching of streamed responses
Date Wed, 04 Sep 2002 07:54:08 GMT
Graham Leggett wrote:

>> For the expiration case, there's a much easier solution than 
>> shadowing the
>> incomplete response.  Add a new state for cache entries: 
>> "being_updated."
>> When you get a request for a cached object that's past its expiration 
>> date,
>> set the cache entry's state to "being_updated" and start retrieving 
>> the new
>> content.  Meanwhile, as other threads handle requests for the same 
>> object,
>> they check the state of the cache entry and, because it's currently 
>> being
>> updated, they deliver the old copy from the cache rather than 
>> dispatching
>> the request to the backend system that's already working on a different
>> instance of the same request.  As long as the thread that's getting 
>> the new
>> content can replace the old content with the new content atomically, 
>> there's
>> no reason to make any other threads wait for the new content.
> Hmmm... ok, not a bad idea, however I see a few hassles though.
> What if a user force-reloads the page. In theory the cached copy 
> should be expired immediately - but here we don't, because shadow 
> threads need to access the cached content in the meantime.

I actually think we *shouldn't* expire the cached copy in this
case.  I want the server, rather than the client, to be able to
decide whether to expire a cache entry.  If the client gets to
dictate the expiration semantics, then we're vulnerable to a
DoS attack (all a malicious client would need to do is start
submitting lots of requests that invalidate cache entries).

> Also - there will be a load spike until the first page is cached (in 
> the case where no previous cache existed).

Yep.  I think the ideal solution would be to combine this
technique with shadowing support.  Then mod_cache could do
one of four things depending on the cache state:
  * cache hit, normal case: deliver the object from cache
  * cache hit, while another thread is updating: deliver the old object
  * cache miss, normal case: go retrieve the content
  * cache miss, while another thread is updating: shadow the response in 

> What happens if an attempt to update an expired cached page hangs? The 
> proxy could find itself serving stale content for a long time while a 
> timeout occurs.

One solution is to add a timestamp for the "being updated" flag.
If another thread sees that the flag is set but the timestamp is
more than 'n' seconds old, that thread sets the timestamp to the
current time and tries to retrieve the object itself.  (This
logic just needs to be atomic so that we don't end up with multiple
threads all deciding to retrieve the content at once.)

>>  * It's going to take a while to make the shadowing work
>>    (due to all the race conditions that have to be addressed).
> I don't see any significant race conditions though.
> All that needs to happen is that all cached responses are 
> readable-from-the-cache the moment they are created, rather than the 
> moment the download is complete. A flag against the cache entry marks 
> it as "still busy". If this flag is set, shadowed threads know to keep 
> waiting for more data appearing in the cache. If the flag is not set, 
> the shadow thread is a normal CACHE_OUT case, and finishes up the 
> request knowing it to be complete.

The race conditions arise in cases where, for example, we find out
halfway through a streamed request that it isn't cacheable after all
because it's just exceeded the max cache size.  Therefore we won't
be caching this response permanently, but we do need to keep the
buffered buckets around until all threads that are shadowing the
response have finished using those buckets.  And then, after the
last of those threads finishes using the data, we need to delete
the saved brigade.  But if the response is *really* large, we can't
keep the whole thing in memory until the last thread finishes
delivering it.  Instead, we'd have to incrementally delete buckets
from the start of the brigade as all the threads finish dealing
with each one.  (And keep new threads from shadowing the response
from the moment when we first delete a bucket.)  This can all be
done with reference counts and careful synchronization, but it's
going to be challenging to debug.


View raw message