Mailing-List: contact dev-help@cocoon.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cocoon.apache.org
Message-ID: <40449AD7.7040402@leverageweb.com>
Date: Tue, 02 Mar 2004 09:31:51 -0500
From: Geoff Howard <cocoon@leverageweb.com>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US;
 rv:1.6) Gecko/20040113
MIME-Version: 1.0
To: dev@cocoon.apache.org
Subject: Re: Event caching and CachedSource
References: <C726734B5ACA2445B2BB93E87ADD5033AF3A28@akxch03.tvnzad.tvnz.co.n
	z> <40448E83.2020404@hippo.nl> <40449288.7090608@leverageweb.com>
 <4044951D.8030306@hippo.nl>
In-Reply-To: <4044951D.8030306@hippo.nl>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Unico Hommes wrote:

> Geoff Howard wrote:
>
>> Unico Hommes wrote:
>>
>>> Corin Moss wrote:
>>>
>>>> Hiya,
>>>>
>>>> I'm probably wrong here, but my understanding of the RefresherImpl is
>>>> that the "timeout" is used to cache the page on a timed basis a la 
>>>> cron
>>>> (although that could be what you mean).
>>>>
>>>> I'm not entirely sure how this helps with external validation directly
>>>> :)
>>>>
>>>> What I've been playing around with in this class is a 
>>>> "refreshInFuture"
>>>> method which does a one-time-only refresh in x seconds (probably 1
>>>> minimum to be safe from expiry problems :)
>>>>
>>>> Does that help you at all? I'm happy to contribute it if it would.
>>>>
>>>>  
>>>>
>>>
>>> Hmm, I guess my explanation was a little bit dense. Let me put this 
>>> into context of the things I am trying to solve.
>>>
>>> We recently deployed a website that is backed by a webdav 
>>> repository. Obviously this introduces some network overhead and 
>>> particular parts of the site can get slow to generate. Especially 
>>> those generated using TraversableGenerators.
>>>
>>> Traditional caching requires the objects on behalf of which it 
>>> caches to provide a so called Validity object in order to determine 
>>> whether its cached objects are still valid upon subsequent request. 
>>> Most sources will try to determine this by providing a last modifed 
>>> timestamp the cache can compare. Since retrieving the last 
>>> modification time is an expensive operation in the case of a webdav 
>>> source determining the validity of a cached response would be almost 
>>> as expensive as generating a new response, requiring a webdav 
>>> propfind for each source that is a member of the generated pipeline.
>>>
>>> Instead we employ a different strategy altoghether. We just tell the 
>>> Cache that the source is always valid using a special Validity 
>>> object. Cache invalidation will be accomplished by an external event 
>>> triggering the removal of all pipelines the Source is associated 
>>> with. This means though that a subsequent request will be slow since 
>>> nothing is cached anymore.
>>>
>>> Perhaps even more importantly, since the pipelines can be huge 
>>> objects the generation of which potentially requires many network 
>>> calls it is much better to cache objects at the most atomic level: 
>>> the source. Hence my interest in CachedSource.
>>>
>>> What I am proposing is to extend the capability of CachedSource so 
>>> that an external event (say, someone saving a document in the webdav 
>>> repository) will trigger the retrieval of a fresh one. But in the 
>>> background, away from the critical path (asynchronous).
>>>
>>> Hope that explains it better.
>>
>>
>> Ok, I'm done being dense - I get this now. 
>
>
> Sorry it wasn't meant to suggest you or anybody was being dense. Just 
> that my explanation was overly concise.;-)


No, I saw where you placed blame (on your explanation) -- I just 
disagreed with your diagnosis.  :)  In my case, it was the "explainee" 
not the explanation that was the problem!

>> My last response about pluggable cache strategies at the pipeline 
>> level is totally mismatched.  At the Source level you don't have to 
>> worry about re-assembling the pipeline, so all you would have to do 
>> is re-cache the source when it is invalidated externally.
>
>
> Exactly.
>
>> Can you just handle this when the JMS/other event comes in?  
>> Currently it's just translated to an Event and sent to the Cache, but 
>> you could also contact the Source as well at that point?  Where does 
>> the Source cache its data?  In memory in a private member, or in the 
>> Store?
>
>
> You got it! (in the Cache/Store)


Whew!  Ok, I can't give this much more thought ATM, but sounds like a 
good direction to me, for what that's worth.

Geoff