forrest-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tim Williams <william...@gmail.com>
Subject Re: more on locationmap caching
Date Wed, 15 Feb 2006 17:18:52 GMT
On 2/15/06, Ross Gardler <rgardler@apache.org> wrote:
> Tim Williams wrote:
> > On 2/10/06, Tim Williams <williamstw@gmail.com> wrote:
> >
> >>It seems to me that implementing CacheableProcessingComponent with an
> >>input module like LM isn't feasible since there's only one instance of
> >>it and it won't help us cache at the more granular level.  I think I
> >>confirmed that by following it through it's lifecycle this evening.
> >>
> >>Anyway, Ross is on to correcting the validity issue so I thought I'd
> >>spend some time on figuring out how to get away from our
> >>homegrown-hashmap-cache.  I think instead of trying to use the cocoon
> >>cache, the answer is to manage validity ourselves and go directly to
> >>the store.    This gets our little lm cache "managed" with the real
> >>cocoon store as i think it should be.
> >>
> >>The only problem that I can foresee is our current transient store is set with:
> >>
> >><parameter name="maxobjects" value="100"/>
> >>
> >>100 seems extremely small to me anyway but I think if we started to
> >>use it for the lm, we'd find that it's definitely too small and would
> >>spend as much resources cleaning itself as to make it not worth it.
> >>Anyone know why it shouldn't be larger?
> >>
> >>Does storing lm cached hints in the transient store seem reasonable?
> >>
> >>Thanks,
> >>--tim
> >
> >
> >
> > FYI. I'm not very keen on implementing it yet, but we may be forced to
> > decide just how important this is to us.  We'll see if I get any other
> > responses, but I'm not holding my breath...
> >
> > http://marc.theaimsgroup.com/?t=113995323300001&r=1&w=2
>
> Thanks for following through Tim.
>
> I've been looking at the use of MultiSourceValidity. It's not trivial
> unfortunately. The way mounted sitemaps are handled is causing problems
> (simply put they are handled differently depending on whether they are
> in a select or not). We need to refactor the handling of mounted
> locationmaps.

By "handled" I gather you mean created and ran through the lifecycle. 
This is a result of our implementing the locationmap grammar "inline".
 In other words, we don't have a grammar and a treeprocessor with
differentiation between nodes/nodebuilders.  Each node is responsible
for building and instantiating its allowable child nodes.  When I
first started looking into this stuff I considered whether a more
robust approach (a la sitemap) should be taken or not but was
[probably rightly] put off [1].  I suspect that you're seeing what I
saw when I wrote that email.

> I've got a solution, but it seems "hacky". I want to find the time to
> look at how Cocoon handles mounted sitemaps and see if we can learn from
> that.

I think any solution is going to "seem" hacky.  Cocoon has the benefit
of a TreeProcessor which does most of the heavy lifting for it.

> With respect to the use of Cocoons store Vs. our current home grown
> solution. I think it is unlikely that we will hit the memory limits
> within the locationmap. The items we are storing are small (strings) and
> the number is relatively low. Even in a very large site it is unlikely
> to go beyond a few thousand items.
>
> If it becomes a problem we can reconsider.

Ok, we should at least comment on this in the source so that if we
ever run into memory issues we know where to look.  For example,
without ?Ron? looking into our current memory consumption, it would
have taken some time to find that the XmlFileModule is the culprit.

> Speaking personally, I need the ability to invalidate the locationmap
> tree when any of the mounted maps are changed. The move to
> MultiSourceValidity will solve this problem. I'm not too concerned right
> now about being able to control the locationmap cache from within the
> sitemap, as long as we can control it from forrest.properties (which we
> already can).

How can we control it from forrest.properties?  Right now, it requires
a restart and it's either on or off in forrest.xconf.

--tim


[1] - http://marc.theaimsgroup.com/?l=xml-cocoon-dev&m=112112847425768&w=2

Mime
View raw message