db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anders Morken <ander...@stud.ntnu.no>
Subject Re: Releasing latches when waiting for locks. When and why?
Date Fri, 08 Dec 2006 09:49:32 GMT
While I'm not an expert on the subject matter, I can't help myself but
chime in and share my thoughts on this... =)

During the course of our project on analyzing Derby's SMP scalability
properties, we've investigated Derby's locking and latching pretty thoroughly.

We've followed this thread with interest as it's very much related to our
ongoing work - and moving latches out of the data structure locks are managed
in is one of the suggestions for improving scalability we have noted in our

Anyway, one thing I've found while working on Derby is that the separation
between interfaces and implementations can be a great advantage - adding the
capability to use Java 1.4's NIO interfaces in RAFContainer was simplified a
lot when we could just overload a few methods with a 1.4 specific class.

Relating this to BasePage and latching, I believe that an implementation
restricted to Java 1.3/1.4's concurrency control abilities will be rather
disadvantaged compared to ReentrantLock and it's friends living in
java.util.concurrent.locks in Java 1.5+.

Now, I realise that I may sound like a broken record now - for DERBY-801 I
jumped on the "new fancy features!" bandwagon, and I'm doing it again now.

However, what I'm trying to convey this time isn't a strong desire to make
something cool with the latest features - but to leave an avenue open for that
at a later stage. This will be a bit harder (and probably a lot messier ;) if
all the latching code is integrated into the BasePage code.

At the very least I'd suggest factoring out latch functionality (state,
synchronization, waiting) into a separate class. The LockFactory interface
could (maybe with some modifications?) still be the factory interface for
latches, even if they are not necessarily managed and stored in the big lock
hash table.

Giving the Lock factory responsibility for picking the appropriate latch
implementation would not only save the BasePage implementation from containing
a lot of latch logic, but it could also permit us to have, say, an
implementation like the current one where deadlocks involving latches and
locks can be debugged together for debugging work, a lightweight pure-java
exclusive latch for Java 1.4 and possibly a shared/exclusive latch using
ReentrantReadWriteLocks from Java 1.5.

Now, I'm just mentioning the last one because we've observed significant wait
times caused by contended latch waits on the root and second-level blocks in a
B-tree index. Adding shared latching for B-tree traversing code could possibly
be one way to alleviate this problem. 

Of course great care must be taken when working in this part of Derby and more
thought should go into this before a decision on how to handle this is made,
as Mike has pointed out it is easy to make a mistake here that causes silent
corruption and data loss. (Not to mention that different latching
implementations on different JVMs and debug/non-debug builds could lead to
some very, uhm, "interesting" Heisenbugs... =)

Anyway, I'm a newbie at this, but those are my two cents, and I'm just
throwing my thoughts out here in the hope that someone finds a use for them.
To reiterate - my point is that if we keep this architecturally clean, we can
reap some benefits right away and keep the avenue open for more cool stuff
later. My ideas for what could be "cool stuff" are not necessarily good ideas.

Anders Morken

My opinions may have changed, but not the fact that I am right!

View raw message