cocoon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sylvain Wallez <sylv...@apache.org>
Subject Re: Fixing store design (long) (was Re: CocoonForms server sizing?)
Date Sun, 07 Dec 2003 10:49:38 GMT
Christian Haul wrote:

> Sylvain Wallez wrote:
>
>> - a transient store: <transient-store> in cocoon.xconf, 
>> Store.TRANSIENT_STORE in Java code
>> - a persistent store: <persistent-store>, Store.PERSISTANT_STORE 
>> which equals Store.ROLE (more on this below)
>>
>> The transient-store, as its name implies, should be transient, and 
>> used to cache objects that should not be serialized, either because 
>> they cannot (the case of Xalan Transformers) or because it doesn't 
>> make sense (e.g. some app-related data that needs to be refreshed on 
>> startup).
>>
>> The persistent store should be used for objects that can be either 
>> long-lived or whose size justify their storage on disk. This includes 
>> of course the CachingPipeline results.
>
>
> Just regarding the names I would expect a persistent store to keep its 
> contents and never evict entries. Maybe even spanning application stop 
> / start. A transient store OTOH could throw out entries at will and 
> maybe should be clean after an application restart. However, if the 
> store uses a disk should not be important. Of course it would be 
> highly desirable to only pass serializable objects (or object trees) 
> to a persistent
> store, maybe even to the transient store.
>
> Really, a transient store is a little like an oxymoron and should be 
> named a cache while a persistent store is, well, a store.


You're right, and actually I think "store" was badly chosen since these 
are actually caches. Even the persistent store may have a limitation 
size limitation, meaning there is no guarantee that objects put in the 
store can be found again later.

> Certainly it would be nice to express an importance of a cache entry 
> for consideration by the cache manager when selecting entries for 
> eviction. But then I'm not quite sure if LRU and entry size are a 
> heuristic good enough.
>
> A persistent store may use a two step design, putting entries into 
> memory before writing eg on a disk. At this point we need to be clear 
> how good the persistence guarantee really is. Since we're not building 
> a database management system, best effort would certainly do.


>> Private caches all over the place
>> ---------------------------------
>> I mentioned above components that pre-analyze files like jxtemplate, 
>> woody form definitions, flowscript, etc. Now if we look closer at 
>> these components, we see that each of them has its own private cache 
>> (often a static Map). This means that every loaded file is kept in 
>> memory forever, even if only used once in the system lifetime, and 
>> even if the corresponding file is actually deleted!
>
>
> This is a big problem, indeed. Since many of those actually build 
> their private cache from a source, I have started a SourceCache in the 
> scratchpad block but got distracted :-( Still, maybe it could be 
> useful when cleaning up the private caches.
>
> The only implementation (SoftSourceCache) uses a ReferenceMap from 
> JCCol that could at least be cleaned by the garbage collector when 
> memory is low. Using a store would surely be better, though. But it's 
> only a first cut :-)


I was thinking to introduce some helper class to handle source-related 
store entries, along the lines of 
XSLTProcessor.TransformerHandlerAndValidity. Your SourceCache goes that 
way, but IMO it should be "just" a helper class rather than a component. 
Also, it manages its own cache ;-)

The ReferenceMap may also be an interesting alternative to the 
MRUMemoryStore.

>> I propose to clearly distinguish the 3 roles and the associtated 
>> semantics:
>> - Store.ROLE is the "general-purpose" store. A component that doesn't 
>> care if the cache is transient or persistent should use this one. 
>> Being general-purpose, it should be efficient but also swap old 
>> objects to persistent storage.
>
>
> I'd say a store should have no guarantees at all, well, beyond "after 
> store, retrieve the result is either an (deep) equal object as stored 
> or null"


Agree.

>> - Store.TRANSIENT_STORE should be used to keep objects that aren't 
>> serializable but should be kept in memory as far as possible. The 
>> flush strategy of this store should not be mixed with a limited-size 
>> MRU policy of a persistent store front-end.
>
>
> Yep.
>
>> - Store.PERSISTANT_STORE should be, as its name implies, only 
>> persistant, with no memory front-end or whatsoever.
>
>
> There is no reason to forbid a memory based stage for a persistent 
> store. But it should be clear whether there is a guarantee in a 
> transactional sense or not. Since this is an Avalon component issue, 
> I'd say there need to be a role for transactional and 
> non-transactional stores. (Ie the first guarantees that the data is 
> persisted when the call returns while the second just guarantees to 
> persist the data upon normal application shutdown and only if no error 
> occurs)


<snip/>

> Please consider the SourceCache idea.


Sure!

Sylvain

-- 
Sylvain Wallez                                  Anyware Technologies
http://www.apache.org/~sylvain           http://www.anyware-tech.com
{ XML, Java, Cocoon, OpenSource }*{ Training, Consulting, Projects }
Orixo, the opensource XML business alliance  -  http://www.orixo.com



Mime
View raw message