cocoon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Geoff Howard <coc...@leverageweb.com>
Subject Re: Turning off default MRU store
Date Sat, 06 Mar 2004 01:37:32 GMT
Corin Moss wrote:

>Hi Guys,
>
>I think the thing to remember here is that it's the Disk Cache
>implementation that has this behaviour, and that it's very easy to use a
>different store type if the user feels it's necessary.  There are other,
>"safer" store implementations we can use - the only problem in this case
>is that the index is kept in memory (which is what makes it so fast.)
>If the user decides that this is inappropriate for them, the change to a
>different store type is very easy to make within the config.
>
>I think that moving to a filesystem store would be counter-productive,
>as the thing that JCS provides is total flexibility of store type.
>
>Just my thoughts :)
>  
>

Ok, I apologize that I just can't look into this myself right now, but 
let me try to state what I perceive to be the things we really need so 
we can narrow down whether this is the right direction to be going:

1)
- we need a high-performance in-memory cache
- we need that cache to be backed by some sort of persistence for two 
reasons
  first, so that cached items aren't lost between server starts
  second so that items bumped off the memory cache can still be 
preserved as the assumption is that pulling a cached item off disk is 
still faster than regenerating the entire pipeline.  for example, the 
most popular 100 items are served in memory, but the rest are on disk. 
(and at shutdown, the 100 in-memory ones go to disk).

** is JCS (properly configured) a good way to meet both of these 
requirements?

2) currently we accomplish both of those transparently with a 
transient-store configured to use a persistent-store.  we have some 
subsystems which use a transient-store configured _not_ to use 
persistence because of application requirements.

** is JCS (properly configured) a good way to meet this requirement?

3) we have one subsystem which goes direct to the persistent-store right 
at shutdown which could just as easily go to the in-memory front-end as 
long as the persistence between restarts still happened.

** is JCS (properly configured) a good way to meet this requirement?

I am not following the details well here about the persistence issue.  
If we don't use the Disk Cache, what other persistent (on disk, survives 
restarts) options are there? 

Geoff




>-----Original Message-----
>From: Unico Hommes [mailto:unico@hippo.nl]
>
>Sent: Saturday, 6 March 2004 3:46 a.m.
>To: dev@cocoon.apache.org
>Subject: Re: Turning off default MRU store
>
>
>Geoff Howard wrote:
>
>  
>
>>Unico Hommes wrote:
>>
>>    
>>
>>>Sylvain Wallez wrote:
>>>
>>>      
>>>
>>>>Unico Hommes wrote:
>>>>
>>>>        
>>>>
>>>>>Unico Hommes wrote:
>>>>>
>>>>>          
>>>>>
>>>>>>Corin Moss wrote:
>>>>>>
>>>>>>            
>>>>>>
>>>>>>>Hi Guys,
>>>>>>>
>>>>>>>I might be getting ahead of myself a bit here, but I'm going to
>>>>>>>try and turn off the default MRU store, in favour of the JCS
>>>>>>>              
>>>>>>>
>
>  
>
>>>>>>>based persistent store.  I'd like to try some tests on
>>>>>>>              
>>>>>>>
>
>  
>
>>>>>>>performance without the default MRU - has anyone else tried
>>>>>>>              
>>>>>>>
>
>  
>
>>>>>>>anything similar? I've simply set the core store's role to point
>>>>>>>              
>>>>>>>
>
>  
>
>>>>>>>to the JCS store implementation.
>>>>>>>
>>>>>>>              
>>>>>>>
>>>>>>I guess I already got ahead of you when I renamed
>>>>>>JCSPersistentStore JCSStore just now :-) (And merged it with the
>>>>>>            
>>>>>>
>
>  
>
>>>>>>AbstractJCSStore BTW). It seems to me that JCS is both and it
>>>>>>            
>>>>>>
>
>  
>
>>>>>>could replace all three stores: DefaultStore, TransientStore and
>>>>>>            
>>>>>>
>
>  
>
>>>>>>PersistentStore.
>>>>>>
>>>>>>            
>>>>>>
>>>>>I have to add that this is not the whole story. We do actually need
>>>>>to distinguish between persistent and transient storage. Some
>>>>>          
>>>>>
>
>  
>
>>>>>components explicitly want to persist some data instead of just
>>>>>          
>>>>>
>
>  
>
>>>>>cache it for speed. But as far as caching is concerned I think it
>>>>>          
>>>>>
>
>  
>
>>>>>we can leave it completely up to JCS.
>>>>>          
>>>>>
>>>>
>>>>
>>>>
>>>>Some components are explicitely using the transient-store to keep
>>>>data that shouldn't (or cannot) be serialized. But AFAIK, no
>>>>        
>>>>
>
>  
>
>>>>component directly uses the persistent-store except the store
>>>>        
>>>>
>itself.
>  
>
>>>To my knowlegde there is one in eventcache block that uses the
>>>PersistentStore to persist some info it needs to recover upon next
>>>      
>>>
>
>  
>
>>>startup.
>>>
>>>It does not look to me as though JCS would fit the PersistentStore
>>>role very well. I was thinking that perhaps. We could have JCSStore
>>>      
>>>
>
>  
>
>>>as a replacement for TransientStore and Store roles and use
>>>      
>>>
>
>  
>
>>>FilesystemStore for the PersistentStore role.
>>>      
>>>
>>
>>Wait a minute.  The original issue that brought JCS to the front as
>>that the persistent store was broken and had license problems.  Are
>>    
>>
>
>  
>
>>you saying that JCS isn't a good candidate for persisting the cached
>>    
>>
>
>  
>
>>info to disk, or that the fact that it by default has an MRU memory
>>    
>>
>
>  
>
>>front-end makes it not map directly to persistent-store role cleanly?
>>
>>    
>>
>IIUC, JCS will always use a memory store in front yes. And it suggests
>
>on the JCS website that the persistence process is not very reliable:
>
>from http://jakarta.apache.org/turbine/jcs/IndexedDiskAuxCache.html :
>
>"When the disk cache is properly shutdown, the memory index is written
>
>to disk and the value file is defragmented. When the cache starts up,
>
>the disk cache can be configured to read or delete the index file. This
>
>provides an unreliable persistence mechanism."
>
>  
>
>>The use of persistent store in eventcache shouldn't be a problem with
>>JCS as long as at shutdown, it persists everything it has in its MRU
>>    
>>
>
>  
>
>>if it hasn't already.  If there is some problem it would be much
>>    
>>
>
>  
>
>>better to just go back to the old serializing persistence for the
>>    
>>
>
>  
>
>>event cache data because the only benefit to putting it in the store
>>    
>>
>
>  
>
>>is that you more or less guaranteed that the events and cached
>>    
>>
>
>  
>
>>responses would be kept together.
>>    
>>
>
>
>Yes, giving up that feature would be too bad.
>
>Unico
>  
>


Mime
View raw message