db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mike Matrigali (JIRA)" <j...@apache.org>
Subject [jira] Updated: (DERBY-2991) Index split deadlock
Date Wed, 03 Dec 2008 23:13:44 GMT

     [ https://issues.apache.org/jira/browse/DERBY-2991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Mike Matrigali updated DERBY-2991:

After reviewing the current patch, this does not look like a change appropriate for a backport.
 It is already 5000 lines of diffs and changes a basic concurrency building block of the btree
code.  Getting rid of the scan lock if it can be done with little or no performance penality
does look like a good feature for a major release.  It will simplify a lot of code, but would
like to see a lot of testing before it gets into an official derby release.  

I do agree this is a good direction.  Some of the code that originally depended on the scan
protection lock no longer needs to as part of work that was later done to support the read
uncommitted isolation level.  Off the top of my head my areas of concern would be that the
following operations work correctly in all combinations without the concurrency protection
provided by the scan protection lock (I think they are ok as they should be protected by latches,
but just get worried removing locking from these operations):
reclaim deleted rows

Another area that might be worth thinking about is to make sure the code that get's previous
key locks for serializable is still right without the scan protection lock.  Need to make
sure an intervening split which previously was not possible does not break this code.

This stuff is hard to test for, I will think about these operations and see if there is any
hidden dependency or if they only got the scan protection lock to enable the scan optimization.

I definitely do worry about having to copy around the full key every time one gives up the
latch.  Given the current store interface we can't keep references so we have to allocate

objects.  This in the worst case can lead to allocation/copies for every index reference in
a query and can quickly add up which was why the additional complicaiton of the scan lock
was added in the first place.  It would be interesting to understand the performance overhead
of the copy vs. the extra search.  As I understand it the proposed optimization would in the
"usual" case as long as the page remained in memory eliminate the scan but would not eliminate
the copy.   Probably the worst case is long keys (maybe multi-part) and maybe datatypes that
require object allocations every time they are copied (like decimal).

> Index split deadlock
> --------------------
>                 Key: DERBY-2991
>                 URL: https://issues.apache.org/jira/browse/DERBY-2991
>             Project: Derby
>          Issue Type: Bug
>          Components: Store
>    Affects Versions:,
>         Environment: Windows XP, Java 6
>            Reporter: Bogdan Calmac
>            Assignee: Knut Anders Hatlen
>         Attachments: d2991-preview-1a.diff, d2991-preview-1a.stat, d2991-preview-1b.diff,
d2991-preview-1b.stat, d2991-preview-1c.diff, d2991-preview-1c.stat, derby.log, InsertSelectDeadlock.java,
Repro2991.java, stacktraces_during_deadlock.txt
> After doing dome research on the mailing list, it appears that the index split deadlock
is a known behaviour, so I will start by describing the theoretical problem first and then
follow with the details of my test case.
> If you have concurrent select and insert transactions on the same table, the observed
locking behaviour is as follows:
>  - the select transaction acquires an S lock on the root block of the index and then
waits for an S lock on some uncommitted row of the insert transaction
>  - the insert transaction acquires X locks on the inserted records and if it needs to
do an index split creates a sub-transaction that tries to acquire an X lock on the root block
of the index
> In summary: INDEX LOCK followed by ROW LOCK + ROW LOCK followed by INDEX LOCK = deadlock
> In the case of my project this is an important issue (lack of concurrency after being
forced to use table level locking) and I would like to contribute to the project and fix this
issue (if possible). I was wondering if someone that knows the code can give me a few pointers
on the implications of this issue:
>  - Is this a limitation of the top-down algorithm used?
>  - Would fixing it require to use a bottom up algorithm for better concurrency (which
is certainly non trivial)?
>  - Trying to break the circular locking above, I would first question why does the select
transaction need to acquire (and hold) a lock on the root block of the index. Would it be
possible to ensure the consistency of the select without locking the index?
> -----
> The attached test (InsertSelectDeadlock.java) tries to simulate a typical data collection
application, it consists of: 
>  - an insert thread that inserts records in batch 
>  - a select thread that 'processes' the records inserted by the other thread: 'select
* from table where id > ?' 
> The derby log provides detail about the deadlock trace and stacktraces_during_deadlock.txt
shows that the inser thread is doing an index split.
> The test was run on and with identical behaviour.
> Thanks,
> Bogdan Calmac.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message