directory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Selcuk AYA <>
Subject Re: TXN WORK: advice needed on how to deail with logical caches
Date Mon, 09 Apr 2012 17:57:08 GMT
On Mon, Apr 9, 2012 at 12:32 AM, Emmanuel Lécharny <> wrote:
> Le 4/8/12 9:16 PM, Selcuk AYA a écrit :
>> I am about to revisit the logical caches issue. My plan is to do the
>> following to handle all these caches in a generic way:
>> - a singe version number is kept for all caches.
> Ok
>> - a thread starting a txn read locks an internal readwrite lock.
>> - when a thread needs to modify a cache, it ugrades its lock to
>> exclusive lock. If it detects a version change during this time, it
>> throws a conflict exception. If no, it bumps up the version number and
>> changes the cache.
>> - After committing, thread releases the lock.
> Ok. Not sure I see all the implications of such a choice atm (it's a bit too
> early in the morning :) but I will give it a second thought later)
>> -If thread aborts its txn, then it notifies interceptors in its
>> interceptor chain of the abort. Any interceptor can then rebuild its
>> cache from what is on disk at this point. I am assuming this is
>> possible for all logical caches.
> As I said, we have to move all the existing cache begind a common
> 'interface' which hides the implementation to the callers. This 'interface'
> could handle the notification.
>> Schema registries are  a kind of cache too. If we apply the above
>> algorithm to these, then we would not need to clone the registries
>> while doing a change.We would do the change, and if txn aborts, schema
>> registries would be built from on disk data again.
> Right now, the Schema registries is handled this way :
> - only add/delete operations are allowed
> - when such an operation is proceeded, we clone the entire registries, apply
> the modification, check that the regisyry is still consistant, and if so, we
> switch the registries (it's a volatile, so it should be ok).
> Now, the registries handling is pretty complex : there are many places where
> we use he registries in the chain to check that some operation data are
> correct. If the rehgistries is changed in the middle of an operation, we
> have no way to inform all the threads using the old version that the data
> have changed.
> We are thinking that the best thing to do is to forbid concurrent
> modifications of the Schema Registries. Ie, when such an operation is
> detected, then we block all the new incoming request, wait for all the
> currently running operations to be performed, and then do the modifications.

Actually this is exactly what I wantrf to do. Getting the exclusive
will quiesce all operations and chaging the registries will be an
exclusive operation.However, this exclusiveness starts from
DefaultOperationManager(that is where we have txn boundaries). I
guess it would be legal to reference registries before entering
operation manager and the chain right? In that case maybe I should let
the cloning stay there for now..

> This is a pretty conservative solution, but I don't see how we can do in any
> other way before seriously rethink the impacts of any other solution...
>> There might be some caches this wont work for.Like if we have an entry
>> cache that sits above partitions, this wont work for this cache purely
>> because of perf reasons. If this is the case for any cache, then that
>> cache has to be made MVCC.
> Yeah, that's probably mandatory. Sadly, the server performance will
> essentially depend on the Entry cache to be present, instead of the page
> cache that we have (deserializing an entry is an extremly costly operation).
> We have to think seriously about having a MVCC cache. May be the small
> experiment I did 2 months ago (MVCC in memory) could be reused ?

Right now JDBM cache is MVCC.
> Thanks Selcuk.
> I'll be MIA today (easter and finishing moving my GF fournitures from her
> place to mine - too old for this shit, damn it !) except late tonite (ie
> 11pm CEST).
> --
> Regards,
> Cordialement,
> Emmanuel Lécharny

View raw message