db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Knut Anders Hatlen <Knut.Hat...@Sun.COM>
Subject Re: Releasing latches when waiting for locks. When and why?
Date Tue, 12 Dec 2006 10:46:30 GMT
Anders Morken <andersmo@stud.ntnu.no> writes:

> While I'm not an expert on the subject matter, I can't help myself but
> chime in and share my thoughts on this... =)
>
> During the course of our project on analyzing Derby's SMP scalability
> properties, we've investigated Derby's locking and latching pretty thoroughly.
>
> We've followed this thread with interest as it's very much related to our
> ongoing work - and moving latches out of the data structure locks are managed
> in is one of the suggestions for improving scalability we have noted in our
> report.
>
> Anyway, one thing I've found while working on Derby is that the separation
> between interfaces and implementations can be a great advantage - adding the
> capability to use Java 1.4's NIO interfaces in RAFContainer was simplified a
> lot when we could just overload a few methods with a 1.4 specific class.
>
> Relating this to BasePage and latching, I believe that an implementation
> restricted to Java 1.3/1.4's concurrency control abilities will be rather
> disadvantaged compared to ReentrantLock and it's friends living in
> java.util.concurrent.locks in Java 1.5+.

Could you please elaborate on this? How are they disadvantaged? Wrt
performance, I would think that there are no disadvantages since
ReentrantLock is built on top of monitors and wait/notify, just like
the latches. It could be that ReentrantLock is easier to use, but I
have my doubts since there are semantic mismatches between latches and
ReentrantLock (for instance, the owner of a latch is a transaction and
the owner of a ReentrantLock is a thread, so nested latches could be a
problem).

> Now, I realise that I may sound like a broken record now - for DERBY-801 I
> jumped on the "new fancy features!" bandwagon, and I'm doing it again now.
>
> However, what I'm trying to convey this time isn't a strong desire to make
> something cool with the latest features - but to leave an avenue open for that
> at a later stage. This will be a bit harder (and probably a lot messier ;) if
> all the latching code is integrated into the BasePage code.

Most of the latching logic is currently in BasePage, and the
DERBY-2107 patch actually removes a lot more code from BasePage than
it adds to it, so I don't think that patch will make it any harder to
use cool features in the future. Since we don't know yet which cool
features we will use in the future, how they will be used, or whether
they will be used at all, I'm afraid that rewriting BasePage to use
some general abstraction could be a lot of extra work on something
that may never be used.

Moving the latching out of the lock manager was a relatively small
change. Since we all like incremental changes, I would say that it's
the first increment, and if someone later wishes to factor out all the
latching code from BasePage, that could be the next increment.

> At the very least I'd suggest factoring out latch functionality (state,
> synchronization, waiting) into a separate class. The LockFactory interface
> could (maybe with some modifications?) still be the factory interface for
> latches, even if they are not necessarily managed and stored in the big lock
> hash table.
>
> Giving the Lock factory responsibility for picking the appropriate latch
> implementation would not only save the BasePage implementation from containing
> a lot of latch logic, but it could also permit us to have, say, an
> implementation like the current one where deadlocks involving latches and
> locks can be debugged together for debugging work, a lightweight pure-java
> exclusive latch for Java 1.4 and possibly a shared/exclusive latch using
> ReentrantReadWriteLocks from Java 1.5.
>
> Now, I'm just mentioning the last one because we've observed significant wait
> times caused by contended latch waits on the root and second-level blocks in a
> B-tree index. Adding shared latching for B-tree traversing code could possibly
> be one way to alleviate this problem. 

But even if we factor out the latching code now, we would still need
to rewrite that code if we some time in the future decide to implement
shared latches. And that would probably be the easy part... ;)

> Of course great care must be taken when working in this part of Derby and more
> thought should go into this before a decision on how to handle this is made,
> as Mike has pointed out it is easy to make a mistake here that causes silent
> corruption and data loss. (Not to mention that different latching
> implementations on different JVMs and debug/non-debug builds could lead to
> some very, uhm, "interesting" Heisenbugs... =)

Yes, I think that's a good argument for not having different latch
implementations depending on build/vm.

> Anyway, I'm a newbie at this, but those are my two cents, and I'm just
> throwing my thoughts out here in the hope that someone finds a use for them.
> To reiterate - my point is that if we keep this architecturally clean, we can
> reap some benefits right away and keep the avenue open for more cool stuff
> later. My ideas for what could be "cool stuff" are not necessarily good ideas.
> =)

Thanks for sharing your thoughts! And please let us know the next time
you get cool ideas, too! :)

-- 
Knut Anders

Mime
View raw message