cocoon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Geoff Howard <coc...@leverageweb.com>
Subject Re: Turning off default MRU store
Date Sat, 06 Mar 2004 14:31:42 GMT
Corin Moss wrote:

>Hiya,
>
>That's a very pertinent set of questions - I'll answer as best I can
>below, the only caveat is that I won't proclaim myself a JCS expert,
>given that I was really only introduced to it a few days ago ;)
>  
>

Ok, fair enough! :)

...

>Ok, I apologize that I just can't look into this myself right now, but
>let me try to state what I perceive to be the things we really need so
>we can narrow down whether this is the right direction to be going:
>
>1)
>- we need a high-performance in-memory cache
>- we need that cache to be backed by some sort of persistence for two
>
>reasons
>  first, so that cached items aren't lost between server starts
>  second so that items bumped off the memory cache can still be
>preserved as the assumption is that pulling a cached item off disk is
>still faster than regenerating the entire pipeline.  for example, the
>most popular 100 items are served in memory, but the rest are on disk.
>(and at shutdown, the 100 in-memory ones go to disk).
>
>** is JCS (properly configured) a good way to meet both of these
>requirements?
>
>CM: Yes.  A JCS store can be configured to use an MRU store (the test
>config provided does just this.) This MRU store is in turn configurable
>- number of objects catered for, what type of store (memory etc.) The
>swapping to disk based store is handled in the same way as Cocoon
>currently does - objects are dropped out of MRU store into diskbased
>store. The only difference is that the "disk based" store can in fact be
>any of the several stores supported by JCS (remote server, disk, JISP
>etc.)
>  
>

Ok.

>At shutdown, things can be configured to store to disk (both store, and
>index.)  The only issue is that in the event of an abnormal shutdown the
>index can be lost (this is only when the index is kept in memory of
>course - other store types which use pure disk based index storage
>wouldn't be effected by this.)
>--
>  
>

Ok, on second thought if this is limited to abnormal shutdown this 
should be OK at least for our use with cached pipelines.  I don't really 
know firsthand the other uses of the Cache.  If they are OK with no 
guarantee of persistence, then I think we're set.

>2) currently we accomplish both of those transparently with a
>transient-store configured to use a persistent-store.  we have some
>subsystems which use a transient-store configured _not_ to use
>persistence because of application requirements.
>
>** is JCS (properly configured) a good way to meet this requirement?
>
>CM: Yes.  You could very easily configure a separate role for transient
>store which would use a different group / region (JCS terms.)  This
>region would be configured to use _only_ memory store, with no support
>for persistence.  This could be accomplished as is, by changing roles
>appropriately, and providing different config in the xconf for the
>transient store.
>--
>  
>

Ok, sounds good too.

>3) we have one subsystem which goes direct to the persistent-store right
>at shutdown which could just as easily go to the in-memory front-end as
>long as the persistence between restarts still happened.
>
>** is JCS (properly configured) a good way to meet this requirement?
>
>CM: Yes.  Again, this could either be written to the default store,
>which would go to MRU, and then disk at shutdown, _or_ a different group
>/ region could be created and configured which would go straight to
>disk, without an in-memory MRU involved.
>--
>  
>

I think this particular case works better going to the default store.  
It's a very minor use (one object, and only at shutdown) and so not 
worth a different cache entirely.

>I am not following the details well here about the persistence issue. 
>If we don't use the Disk Cache, what other persistent (on disk, survives
>restarts) options are there?
>
>CM: As I mentioned before, the only issue which needs more investigation
>is abnormal shutdown with an in-memory index, however, this could
>theoretically be up to the user to decide - if their application
>requires an absolute guarantee of persistence at all times, they could
>configure it to use an on-disk index at all times - this would require
>no class change, just a change in the CCF.
>  
>

Ok, I think this bears bringing this issue up in a new thread (so people 
notice it).  The point I take out of this is that JCS makes it explicit 
that successful persistence is not currently guaranteed (at least in the 
default better-performing configuration).  For all we know, this may 
have been the case with JISP too but just not explicit.  I don't know if 
people have used our Cache as a persistence layer for application data.  
I would not have recommended it before, and definitely would not now if 
we go to JCS.  I'm comfortable with this.

Geoff


Mime
View raw message