From dev-return-56550-apmail-cocoon-dev-archive=cocoon.apache.org@cocoon.apache.org Tue Mar 02 13:52:08 2004 Return-Path: Delivered-To: apmail-cocoon-dev-archive@www.apache.org Received: (qmail 30432 invoked from network); 2 Mar 2004 13:52:07 -0000 Received: from daedalus.apache.org (HELO mail.apache.org) (208.185.179.12) by minotaur-2.apache.org with SMTP; 2 Mar 2004 13:52:07 -0000 Received: (qmail 39807 invoked by uid 500); 2 Mar 2004 13:52:00 -0000 Delivered-To: apmail-cocoon-dev-archive@cocoon.apache.org Received: (qmail 39786 invoked by uid 500); 2 Mar 2004 13:52:00 -0000 Mailing-List: contact dev-help@cocoon.apache.org; run by ezmlm Precedence: bulk list-help: list-unsubscribe: list-post: Reply-To: dev@cocoon.apache.org Delivered-To: mailing list dev@cocoon.apache.org Received: (qmail 39770 invoked from network); 2 Mar 2004 13:52:00 -0000 Received: from unknown (HELO host.leverageweb.com) (64.91.254.192) by daedalus.apache.org with SMTP; 2 Mar 2004 13:52:00 -0000 Received: from va-leesburg-cmts5c-90.chvlva.adelphia.net ([67.21.159.90] helo=leverageweb.com) by host.leverageweb.com with esmtp (Exim 4.24) id 1AyAbO-0008FK-MD for dev@cocoon.apache.org; Tue, 02 Mar 2004 09:10:30 -0500 Message-ID: <40449288.7090608@leverageweb.com> Date: Tue, 02 Mar 2004 08:56:24 -0500 From: Geoff Howard User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.6) Gecko/20040113 X-Accept-Language: en-us, en MIME-Version: 1.0 To: dev@cocoon.apache.org Subject: Re: Event caching and CachedSource References: <40448E83.2020404@hippo.nl> In-Reply-To: <40448E83.2020404@hippo.nl> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - host.leverageweb.com X-AntiAbuse: Original Domain - cocoon.apache.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - leverageweb.com X-Spam-Rating: daedalus.apache.org 1.6.2 0/1000/N X-Spam-Rating: minotaur-2.apache.org 1.6.2 0/1000/N Unico Hommes wrote: > Corin Moss wrote: > >> Hiya, >> >> I'm probably wrong here, but my understanding of the RefresherImpl is >> that the "timeout" is used to cache the page on a timed basis a la cron >> (although that could be what you mean). >> >> I'm not entirely sure how this helps with external validation directly >> :) >> >> What I've been playing around with in this class is a "refreshInFuture" >> method which does a one-time-only refresh in x seconds (probably 1 >> minimum to be safe from expiry problems :) >> >> Does that help you at all? I'm happy to contribute it if it would. >> >> >> > > Hmm, I guess my explanation was a little bit dense. Let me put this > into context of the things I am trying to solve. > > We recently deployed a website that is backed by a webdav repository. > Obviously this introduces some network overhead and particular parts > of the site can get slow to generate. Especially those generated using > TraversableGenerators. > > Traditional caching requires the objects on behalf of which it caches > to provide a so called Validity object in order to determine whether > its cached objects are still valid upon subsequent request. Most > sources will try to determine this by providing a last modifed > timestamp the cache can compare. Since retrieving the last > modification time is an expensive operation in the case of a webdav > source determining the validity of a cached response would be almost > as expensive as generating a new response, requiring a webdav propfind > for each source that is a member of the generated pipeline. > > Instead we employ a different strategy altoghether. We just tell the > Cache that the source is always valid using a special Validity object. > Cache invalidation will be accomplished by an external event > triggering the removal of all pipelines the Source is associated with. > This means though that a subsequent request will be slow since nothing > is cached anymore. > > Perhaps even more importantly, since the pipelines can be huge objects > the generation of which potentially requires many network calls it is > much better to cache objects at the most atomic level: the source. > Hence my interest in CachedSource. > > What I am proposing is to extend the capability of CachedSource so > that an external event (say, someone saving a document in the webdav > repository) will trigger the retrieval of a fresh one. But in the > background, away from the critical path (asynchronous). > > Hope that explains it better. Ok, I'm done being dense - I get this now. My last response about pluggable cache strategies at the pipeline level is totally mismatched. At the Source level you don't have to worry about re-assembling the pipeline, so all you would have to do is re-cache the source when it is invalidated externally. Can you just handle this when the JMS/other event comes in? Currently it's just translated to an Event and sent to the Cache, but you could also contact the Source as well at that point? Where does the Source cache its data? In memory in a private member, or in the Store? Geoff