forrest-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ross Gardler <rgard...@apache.org>
Subject Re: Locationmap Caching Help (FOR-732/711)
Date Thu, 30 Mar 2006 18:03:01 GMT
Tim Williams wrote:
> On 3/30/06, Ross Gardler <rgardler@apache.org> wrote:
> 
>>Tim Williams wrote:
>>
>>>I've been looking into FOR-711 [1]/FOR-732 [2] and I'm in need of some
>>>new thoughts on the subject.  My "fix" for FOR-732 (changes to
>>>locationmap having no effect) has effectively reversed much of the
>>>benefit of caching the responses to begin with.  I'll explain...
>>
>>Sorry for not responding sooner, I've been trying to find the time to do
>>so properly. I'm not finding any time at the moment, so I'll just give
>>an "off the top of my head" response, in the hope that it helps in some
>>small way.
>>
>>
>>>The highlights of what I've done:
>>>o) Created isValid() on AbstractNode.
>>>o) Implemented isValid() on each node as appropriate.
>>>o) LocationMapModule now tests whether caching is turned on &&
>>>isValid() before returning a cached result.
>>>o) If the traversed isValid() returns an invalid result back to the
>>>Locationmapmodule, the cache is flushed.
>>>
>>>The results I'm seeing: Slow, but the correct behavior... did I
>>>mention slow [3]?
>>
>>Hmmm... that's quite a performance hit. I don't really understand why it
>>is so bad with caching turned on. It's probably a really stupid question
>>but are we using the right kind of data structure for the cache data.
> 
> 
> Because it's making ~315 validity tests per request.

Wow. Why is that the case? There are only a handful of locationmap files 
in most projects.

(you answer this further down)


>>One other thought, are we checking all isValid() methods or just the one
>>  for the file in which the cached data was found? There is no need to
>>traverse the cache files *below* the one we currently use for giving a
>>result.
> 
> 
> We currently stop only when an invalid file is found.  Otherwise all
> nodes are traversed.  If we had insight into what file a given hint
> was located in, we could just test that directly without needing to do
> the whole AbstractNode.isValid()-recursive thing.  Inside
> getAttribute() we don't have insight into which locationmap returned
> the value to us.

Therein lies the problem. We should be testing on a per file basis, not 
a per node entry.

>>>So what to do and what am I asking?
>>
>> > ...  even if we were to
>>
>>>implement our own store ... it seems to me it's the sheer number
>>>of accesses that's the issue rather than storage location.
>>
>>...
>>
>>
>>> Having a
>>>configurable timeout (so that validity tests are only done every X
>>>number of minutes) would help but seems ultra-hacky.
>>
>>Why hacky? Isn't it standard practice for a cache to have a configurable
>>TTL, then you use that it to fine tune performance, adjusting the cache
>>on different requrests.
> 
> 
> Yeah, our cache isn't that sophisticated, when I'm talking timeout,
> I'm talking about the *whole* cache (HashMap).  I think it's standard
> practice to evict individual items in the store based on TTL but not
> potentially invalidate the whole thing.

By caching at the file level, rather than the node level we solve this 
problem. We can define a TTL in the LM file itself. For example, there 
is little point in caching the Forrest core locationmaps since they will 
never change in production.

In my (aborted) experiments with the Cocoon MutiFileValidity I had this 
working, to an extent. The problem was with how the locationmaps are 
mounted, it was not possible to create such a validity object with the 
current way the locationmap works (see archives). I've not had time to 
address this yet.

Ross

Ross

Mime
View raw message