cocoon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexander Klimetschek <>
Subject Re: Broken caching of servlet: source in some cases
Date Sat, 21 Apr 2007 20:54:51 GMT
Grzegorz Kossakowski schrieb:
>> I've talked with Reinhard about this issue and we agreed that old Last-Modified is
needed when validity is returned. We'll have to maintain
>> our own store with these values.
>> I'm working on the definitive solution to this problem.
> Done. Could you give it a try?
> Changes are described in comments of source code and svn log.

Great! I can try it on Monday, have to do other things this weekend.

But I found out two other problems with the order of methods called on the 

1) A (Resource)Reader will always call getLastModified() before 
getValidity(), which breaks the caching completely, since it starts a 
servlet connection without the If-Modified-Since set. But it looks like this 
could be fixed with your new changes! My first idea was to return -1 in 
getLastModified() until the real value is known after the connection was 
executed. But I am not sure if this will break other use cases.

(BTW: The method ServletConnection.connect() should be renamed to call() or 
execute() - connect sounds like doing only the first step, "establishing a 
connection", but it actually connects, gets the data and "closes" the 

2) The other problem happens when the validity will be integrated inside an 
AggregatedValidity together with others, eg. when using <map:aggregate />. 
In that case it is possible that although the source validity returns valid 
(and has no response data), the pipeline calls getInputStream(). This is 
when the other validities are invalid and the decision is made to retrieve 
fresh new data from all sources. That was the mysterious last bug ;-)

For this I would propose to change the getInputStream() implementation that 
it will do a connection without if-modified-since header set regardless if 
there already was a connection (started from isValid method). This will end 
in two full sitemap processings, but there seems no other solution to me.



I evaluated the entire caching algorithms in Cocoon during debugging and 
here are all the important bits and pieces I came up with from the point of 
a Source developer. Some is noted on but not 
everything, so I'd like to share it on the list for future work:

Sources & Caching in Cocoon

This is typical order of org.apache.excalibur.source.Source and 
SourceValidity methods called regarding caching:

getURI()  <- used as cache key for the cached response + the cached validity

getLastModified()  <- called only by ResourceReader to set the Last-Modified
                       header if the value is > 0

SourceValidity.isValid()  <- called on cached (old) validity if found
                              in cache

getValidity()  <- called if the old cached validity returned 0 (UNKNOWN) on
                   isValid() or for putting the new data into the cache

                <- called on cached validity with the new validity as

getInputStream()  <- called when any isValid() method returned -a (INVALID)
                      but also when some other information outside the
                      current source forces new data to be fetched (eg.
                      when SourceValidity is put into an AggregatedValidity
                      together with others - one invalid validity makes all
                      sources invalid!)

If the isValid(SourceValidity) method returns UNKNOWN, the new validity will 
be refetched, so getValidity() is called a second time (!).

Alexander Klimetschek

View raw message