cocoon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Unico Hommes" <Un...@hippo.nl>
Subject RE: Accessing cache validities from flow
Date Tue, 16 Dec 2003 14:15:38 GMT
 

Stefano Mazzocchi wrote:
> 
> After lots of thinking, I came to the conclusion that the 
> only way of doing serious caching is using the "inverted 
> cache" approach, which is prototyped in the "eventcache" block.
> 
> My problem is that linotype is too slow for the load that my 
> blog generates and runs at 98% of server CPU. The reason for 
> this is that very little caching is performed. I could 
> pregenerate the pages and let them served static, but that 
> would prevent me from playing with run-time data like the 
> referring request headers.
> 
> Now, the core of linotype is the following pipeline:
> 
>   1) scans the directories that hold the news
>   2) includes them
>   3) filters them
> 
> as you can easily see, the approach is totally wrong: 
> filtering should happen *before* the inclusion... the problem 
> is that since the file system is such a stupid repository, 
> metadata is included in the files, so I have to include them 
> before I can filter them.
> 
> The use of a serious repository and a serious query language 
> on top of the metadata will solve that issue and allow to 
> scale a few orders of magnitue more in terms of requests, 
> which will keep me happy for a while.
> 
> But still, thinking a little bigger than my own blog, the 
> ideal solution would be to avoid querying entirely if there 
> is no need to.
> 
> Now, the way the event cache works is like this:
> 
>   a) a cache validity is generated
>   b) pipeline is executed
>   c) result is stored in the cache
> 
> then the pipeline is never called, until an event is 
> triggered externally (from an avalon component) that 
> invalidates that particular cache entity.
> 
> Now, the problem is: if I am not the one who generates the 
> validity (in linotype, it's the directory generator), how can 
> I invalidate it? how can I have access to it?
> 
> I'm starting to think that the cocoon cache needs a serious 
> redesign to deal with these issues... and this scares me :-( 
> but it's *waay* too important to let go.
> 

I have been thinking about this issue for a while. The event cache
introduces the problem of where to define the cache strategy. The use of
a wrapper sitemap component such as EventCacheGenerator feels like more
of a hack then a real solution. Perhaps some kind of AoP mechanism is
needed?

Currently the caching strategy is defined as the aggregation of validity
objects harvested from the individual pipeline components. This works in
most situations but we need a way to overide the strategy at a different
level for things such as event caching and timeout overides to work
nicely.

As I am extrememly busy at the moment I won't have time to continue this
discussion now, but just so that you know I'm really interested in this
and I really like to discuss this more.

Later,
Unico

Mime
View raw message