httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "" <>
Subject Re: mod-cache-requestor plan
Date Mon, 11 Jul 2005 08:47:50 GMT
Hi Parin
I'm a newbie too and I was preparing a clear_cache command in order to force cleaning of some
desired path.
Your idea is very good.
My personal considerations are:
1 - you have to avoid multiple requests to the back end: if the page is very frequently clicked
your mechanism can bring the system to perform many regeneration requests to the back end,
so maybe the best is to use a queue to purge duplicated requests
2 - if you already know the expiry time of the pages (when the page is inserted into the queue
you know) you could "book" the request for the regeneration instead of waiting for a request
close to the expiration
3 - it could be useful to perform regeneration just of "highly clicked" URLs, introducing
a heuristic to define what's a "highly clicked" URL in way similar to swapped-out pages from
memory in virtual memory management



> Da: Parin Shah <>
> Data: Sun, 10 Jul 2005 23:24:10 -0500
> A:
> Oggetto: mod-cache-requestor plan
> Hi All,
> I am a newbie. I am going to work on mod-cache and a new module
> mod-cache-requester as a part of Soc program.
> Small description of the module is as follows.
> When the page expires from the cache, it is removed from cache and
> thus next request has to wait until that page is reloaded by the
> back-end server. But if we add one more module which re-request the
> soon-to-expire pages, in that case such pages wont be removed from the
> cache and thus would reduce the response time.
> Here is the overview of how am I planning to implement it.
> 1. when a page is requested and it exists in the cache, mod_cache
> checks the expiry time of the page.
> 2. If (expiry time – current time)  < Some_Constant_Value,
> then mod-cache notifies mod_cache_requester about this page. 
> This communication between mod_cache and mod_cache_requester should
> incur least overhead as this would affect current request's response
> time.
> 3. mod_cache_requester will re-request the page which is soon-to-expire.
> Each such request is done through separate thread so that multiple
> pages could be re-requested simultaneously.
> This request would force the server to reload the content of the page
> into the cache even if it is already there. (this would reset the
> expiry time of the page and thus it would be able to stay in the cache
> for longer duration.)
> Please let me know what you think about this module. Also I have some
> questions  and your help would be really useful.
> 1.what would be the best way for communication between mod_cache and
> mod_cache_requester.  I believe that keeping  mod_cache_requester in a
> separate thread would be the best way.
> 2.How should the mod_cache_requester send the re-request to the main
> server. I believe that sending it as if the request has come from the
> some client would be the best way to implement. But we need to attach
> some special status with this request so that cache_lookup  is
> bypassed and output_filter is not added as we dont need to stream the
> output.
> 3.Other than these questions, any suggestion/correction is welcome.
> Any pointers to the details of related modules( mod-cache,
> communication between mod-cache and backend server) would be helpful
> too.
> Thanks,
> Parin.

View raw message