cocoon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Corin Moss" <Corin.M...@tvnz.co.nz>
Subject RE: Event caching and CachedSource
Date Tue, 02 Mar 2004 13:47:41 GMT

Hi,

That makes perfect sense. I implemented exactly that this week, although
I have to admit it is nowhere near as elegant as I would like.

Basically in my case it's a database update / delete / insert triggering
a cache clear, it is then a specific request generating the re-cache.

I'm then employing something similar to the "I'm sorry" method mentioned
previously.  I've re-implemented the "test" cron job, using a new method
on RefresherImpl which as I mentioned before, caches in the future (by
adding a new scheduled job.) 

Effectively the request to trigger the re-cache might take a few
milliseconds, but the job to recache the page goes on in the background
for as longs as it takes.

At this point, it is simply a call to an internal URL on loopback (very
messy.)  However, I see no reason that something like the XSPUtil
include source couldn't be used.  It would also pay to have a look at
the BackgroundEnvironment - its only weakness is (as commented) that it
doesn't support objects which try to access specific HTTPenvironment
things (object maps etc.)

Like I said - I've got this working at the moment - let me know if you
want the code :)

Corin


-----Original Message-----
From: Unico Hommes [mailto:unico@hippo.nl]
Sent: Wednesday, 3 March 2004 2:39 a.m.
To: dev@cocoon.apache.org
Subject: Re: Event caching and CachedSource


Corin Moss wrote:

>Hiya,
>
>I'm probably wrong here, but my understanding of the RefresherImpl is
>that the "timeout" is used to cache the page on a timed basis a la cron

>(although that could be what you mean).
>
>I'm not entirely sure how this helps with external validation directly
>:)
>
>What I've been playing around with in this class is a "refreshInFuture"

>method which does a one-time-only refresh in x seconds (probably 1
>minimum to be safe from expiry problems :)
>
>Does that help you at all? I'm happy to contribute it if it would.
>
> 
>

Hmm, I guess my explanation was a little bit dense. Let me put this into

context of the things I am trying to solve.

We recently deployed a website that is backed by a webdav repository.
Obviously this introduces some network overhead and particular parts of
the site can get slow to generate. Especially those generated using
TraversableGenerators.

Traditional caching requires the objects on behalf of which it caches to

provide a so called Validity object in order to determine whether its
cached objects are still valid upon subsequent request. Most sources
will try to determine this by providing a last modifed timestamp the
cache can compare. Since retrieving the last modification time is an
expensive operation in the case of a webdav source determining the
validity of a cached response would be almost as expensive as generating

a new response, requiring a webdav propfind for each source that is a
member of the generated pipeline.

Instead we employ a different strategy altoghether. We just tell the
Cache that the source is always valid using a special Validity object.
Cache invalidation will be accomplished by an external event triggering
the removal of all pipelines the Source is associated with. This means
though that a subsequent request will be slow since nothing is cached
anymore.

Perhaps even more importantly, since the pipelines can be huge objects
the generation of which potentially requires many network calls it is
much better to cache objects at the most atomic level: the source. Hence

my interest in CachedSource.

What I am proposing is to extend the capability of CachedSource so that
an external event (say, someone saving a document in the webdav
repository) will trigger the retrieval of a fresh one. But in the
background, away from the critical path (asynchronous).

Hope that explains it better.

Unico

================================================================
CAUTION: This e-mail and any attachment(s) contains information
that is intended to be read only by the named recipient(s). It
may contain information that is confidential, proprietary or the
subject of legal privilege. This information is not to be used by
any other person and/or organisation. If you are not the intended
recipient, please advise us immediately and delete this e-mail
from your system. Do not use any information contained in it.

================================================================
For more information on the Television New Zealand Group, visit
us online at http://www.tvnz.co.nz
================================================================

Mime
View raw message