cocoon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Geoff Howard <>
Subject Re: Event caching and CachedSource
Date Tue, 02 Mar 2004 14:31:51 GMT
Unico Hommes wrote:

> Geoff Howard wrote:
>> Unico Hommes wrote:
>>> Corin Moss wrote:
>>>> Hiya,
>>>> I'm probably wrong here, but my understanding of the RefresherImpl is
>>>> that the "timeout" is used to cache the page on a timed basis a la 
>>>> cron
>>>> (although that could be what you mean).
>>>> I'm not entirely sure how this helps with external validation directly
>>>> :)
>>>> What I've been playing around with in this class is a 
>>>> "refreshInFuture"
>>>> method which does a one-time-only refresh in x seconds (probably 1
>>>> minimum to be safe from expiry problems :)
>>>> Does that help you at all? I'm happy to contribute it if it would.
>>> Hmm, I guess my explanation was a little bit dense. Let me put this 
>>> into context of the things I am trying to solve.
>>> We recently deployed a website that is backed by a webdav 
>>> repository. Obviously this introduces some network overhead and 
>>> particular parts of the site can get slow to generate. Especially 
>>> those generated using TraversableGenerators.
>>> Traditional caching requires the objects on behalf of which it 
>>> caches to provide a so called Validity object in order to determine 
>>> whether its cached objects are still valid upon subsequent request. 
>>> Most sources will try to determine this by providing a last modifed 
>>> timestamp the cache can compare. Since retrieving the last 
>>> modification time is an expensive operation in the case of a webdav 
>>> source determining the validity of a cached response would be almost 
>>> as expensive as generating a new response, requiring a webdav 
>>> propfind for each source that is a member of the generated pipeline.
>>> Instead we employ a different strategy altoghether. We just tell the 
>>> Cache that the source is always valid using a special Validity 
>>> object. Cache invalidation will be accomplished by an external event 
>>> triggering the removal of all pipelines the Source is associated 
>>> with. This means though that a subsequent request will be slow since 
>>> nothing is cached anymore.
>>> Perhaps even more importantly, since the pipelines can be huge 
>>> objects the generation of which potentially requires many network 
>>> calls it is much better to cache objects at the most atomic level: 
>>> the source. Hence my interest in CachedSource.
>>> What I am proposing is to extend the capability of CachedSource so 
>>> that an external event (say, someone saving a document in the webdav 
>>> repository) will trigger the retrieval of a fresh one. But in the 
>>> background, away from the critical path (asynchronous).
>>> Hope that explains it better.
>> Ok, I'm done being dense - I get this now. 
> Sorry it wasn't meant to suggest you or anybody was being dense. Just 
> that my explanation was overly concise.;-)

No, I saw where you placed blame (on your explanation) -- I just 
disagreed with your diagnosis.  :)  In my case, it was the "explainee" 
not the explanation that was the problem!

>> My last response about pluggable cache strategies at the pipeline 
>> level is totally mismatched.  At the Source level you don't have to 
>> worry about re-assembling the pipeline, so all you would have to do 
>> is re-cache the source when it is invalidated externally.
> Exactly.
>> Can you just handle this when the JMS/other event comes in?  
>> Currently it's just translated to an Event and sent to the Cache, but 
>> you could also contact the Source as well at that point?  Where does 
>> the Source cache its data?  In memory in a private member, or in the 
>> Store?
> You got it! (in the Cache/Store)

Whew!  Ok, I can't give this much more thought ATM, but sounds like a 
good direction to me, for what that's worth.


View raw message