directory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Emmanuel Lécharny <>
Subject Re: TXN WORK: advice needed on how to deail with logical caches
Date Mon, 09 Apr 2012 07:32:14 GMT
Le 4/8/12 9:16 PM, Selcuk AYA a écrit :
> I am about to revisit the logical caches issue. My plan is to do the
> following to handle all these caches in a generic way:
> - a singe version number is kept for all caches.
> - a thread starting a txn read locks an internal readwrite lock.
> - when a thread needs to modify a cache, it ugrades its lock to
> exclusive lock. If it detects a version change during this time, it
> throws a conflict exception. If no, it bumps up the version number and
> changes the cache.
> - After committing, thread releases the lock.
Ok. Not sure I see all the implications of such a choice atm (it's a bit 
too early in the morning :) but I will give it a second thought later)
> -If thread aborts its txn, then it notifies interceptors in its
> interceptor chain of the abort. Any interceptor can then rebuild its
> cache from what is on disk at this point. I am assuming this is
> possible for all logical caches.
As I said, we have to move all the existing cache begind a common 
'interface' which hides the implementation to the callers. This 
'interface' could handle the notification.
> Schema registries are  a kind of cache too. If we apply the above
> algorithm to these, then we would not need to clone the registries
> while doing a change.We would do the change, and if txn aborts, schema
> registries would be built from on disk data again.
Right now, the Schema registries is handled this way :
- only add/delete operations are allowed
- when such an operation is proceeded, we clone the entire registries, 
apply the modification, check that the regisyry is still consistant, and 
if so, we switch the registries (it's a volatile, so it should be ok).

Now, the registries handling is pretty complex : there are many places 
where we use he registries in the chain to check that some operation 
data are correct. If the rehgistries is changed in the middle of an 
operation, we have no way to inform all the threads using the old 
version that the data have changed.

We are thinking that the best thing to do is to forbid concurrent 
modifications of the Schema Registries. Ie, when such an operation is 
detected, then we block all the new incoming request, wait for all the 
currently running operations to be performed, and then do the modifications.

This is a pretty conservative solution, but I don't see how we can do in 
any other way before seriously rethink the impacts of any other solution...
> There might be some caches this wont work for.Like if we have an entry
> cache that sits above partitions, this wont work for this cache purely
> because of perf reasons. If this is the case for any cache, then that
> cache has to be made MVCC.
Yeah, that's probably mandatory. Sadly, the server performance will 
essentially depend on the Entry cache to be present, instead of the page 
cache that we have (deserializing an entry is an extremly costly operation).

We have to think seriously about having a MVCC cache. May be the small 
experiment I did 2 months ago (MVCC in memory) could be reused ?

Thanks Selcuk.

I'll be MIA today (easter and finishing moving my GF fournitures from 
her place to mine - too old for this shit, damn it !) except late tonite 
(ie 11pm CEST).

Emmanuel Lécharny

View raw message