httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ian Holsman <>
Subject Re: [PATCH] mod_cache: support caching of streamed responses
Date Wed, 04 Sep 2002 15:13:05 GMT
Brian Pane wrote:
> Graham Leggett wrote:
>>> For the expiration case, there's a much easier solution than 
>>> shadowing the
>>> incomplete response.  Add a new state for cache entries: 
>>> "being_updated."
>>> When you get a request for a cached object that's past its expiration 
>>> date,
>>> set the cache entry's state to "being_updated" and start retrieving 
>>> the new
>>> content.  Meanwhile, as other threads handle requests for the same 
>>> object,
>>> they check the state of the cache entry and, because it's currently 
>>> being
>>> updated, they deliver the old copy from the cache rather than 
>>> dispatching
>>> the request to the backend system that's already working on a different
>>> instance of the same request.  As long as the thread that's getting 
>>> the new
>>> content can replace the old content with the new content atomically, 
>>> there's
>>> no reason to make any other threads wait for the new content.
>> Hmmm... ok, not a bad idea, however I see a few hassles though.
>> What if a user force-reloads the page. In theory the cached copy 
>> should be expired immediately - but here we don't, because shadow 
>> threads need to access the cached content in the meantime.
> I actually think we *shouldn't* expire the cached copy in this
> case.  I want the server, rather than the client, to be able to
> decide whether to expire a cache entry.  If the client gets to
> dictate the expiration semantics, then we're vulnerable to a
> DoS attack (all a malicious client would need to do is start
> submitting lots of requests that invalidate cache entries).
this is not RFC compliant, so some people will prefer RFC compliance.
to give the more sane (imho ;-) use case, we have a directive which
will ignore the client cache request.

>> Also - there will be a load spike until the first page is cached (in 
>> the case where no previous cache existed).

> Yep.  I think the ideal solution would be to combine this
> technique with shadowing support.  Then mod_cache could do
> one of four things depending on the cache state:
>  * cache hit, normal case: deliver the object from cache
>  * cache hit, while another thread is updating: deliver the old object
>  * cache miss, normal case: go retrieve the content
>  * cache miss, while another thread is updating: shadow the response in 
> progress
>> What happens if an attempt to update an expired cached page hangs? The 
>> proxy could find itself serving stale content for a long time while a 
>> timeout occurs.
> One solution is to add a timestamp for the "being updated" flag.
> If another thread sees that the flag is set but the timestamp is
> more than 'n' seconds old, that thread sets the timestamp to the
> current time and tries to retrieve the object itself.  (This
> logic just needs to be atomic so that we don't end up with multiple
> threads all deciding to retrieve the content at once.)

I was actually thinking that we could use another priority queue
and a seperate thread(s) to call the back end, limiting the amount of 
traffic which can ever get to the backend at all.
>>>  * It's going to take a while to make the shadowing work
>>>    (due to all the race conditions that have to be addressed).
>> I don't see any significant race conditions though.
>> All that needs to happen is that all cached responses are 
>> readable-from-the-cache the moment they are created, rather than the 
>> moment the download is complete. A flag against the cache entry marks 
>> it as "still busy". If this flag is set, shadowed threads know to keep 
>> waiting for more data appearing in the cache. If the flag is not set, 
>> the shadow thread is a normal CACHE_OUT case, and finishes up the 
>> request knowing it to be complete.
> The race conditions arise in cases where, for example, we find out
> halfway through a streamed request that it isn't cacheable after all
> because it's just exceeded the max cache size.  Therefore we won't
> be caching this response permanently, but we do need to keep the
> buffered buckets around until all threads that are shadowing the
> response have finished using those buckets.  And then, after the
> last of those threads finishes using the data, we need to delete
> the saved brigade.  But if the response is *really* large, we can't
> keep the whole thing in memory until the last thread finishes
> delivering it.  Instead, we'd have to incrementally delete buckets
> from the start of the brigade as all the threads finish dealing
> with each one.  (And keep new threads from shadowing the response
> from the moment when we first delete a bucket.)  This can all be
> done with reference counts and careful synchronization, but it's
> going to be challenging to debug.
> Brian

View raw message