Hi Kristian,

I've attached our Derby log file that has all the threads that were interrupted when the problem occurred (quite a few).

Our application was in the middle of a user initiated query when the user started another operation. For performance reasons we call shutdown() on the previous thread pool which causes the interrupt() call on all the threads. As you can see in the stack traces some threads are querying Lucene and some are querying Derby.

Mostly we just want to cancel the current query but from the higher levels of the application its not apparent if a thread is processing a Lucene or a Derby query.


Thanks again,

Luke


On Mon, Jun 21, 2010 at 6:52 PM, Kristian Waagan <Kristian.Waagan@sun.com> wrote:
On 21.06.10 09:20, Luke Quinane wrote:
Hi all,

We've seen a problem today where we have several threads querying our database and when one gets interrupted the others are stuck waiting for a lock. Here is the stack trace for the stuck thread(s):

daemon prio=4
"DefaultExecutorService-pool-1-thread-47" Id=98 WAITING on org.apache.derby.impl.services.locks.ActiveLock@6e6f45a1
at java.lang.Object.wait(Native Method)
-  waiting on org.apache.derby.impl.services.locks.ActiveLock@6e6f45a1
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.services.locks.ActiveLock.waitForGrant(ActiveLock.java:115)
at org.apache.derby.impl.services.locks.ConcurrentLockSet.lockObject(ConcurrentLockSet.java:463)
at org.apache.derby.impl.services.locks.ConcurrentLockSet.zeroDurationLockObject(ConcurrentLockSet.java:855)
at org.apache.derby.impl.services.locks.AbstractPool.zeroDurationlockObject(AbstractPool.java:297)
at org.apache.derby.impl.services.locks.ConcurrentPool.zeroDurationlockObject(ConcurrentPool.java:28)
at org.apache.derby.impl.store.raw.xact.RowLocking2nohold.lockRecordForRead(RowLocking2nohold.java:89)
at org.apache.derby.impl.store.access.heap.HeapController.lockRow(HeapController.java:520)
at org.apache.derby.impl.store.access.heap.HeapController.lockRow(HeapController.java:638)
at org.apache.derby.impl.store.access.btree.index.B2IRowLocking3.lockRowOnPage(B2IRowLocking3.java:309)
at org.apache.derby.impl.store.access.btree.index.B2IRowLocking3._lockScanRow(B2IRowLocking3.java:599)
at org.apache.derby.impl.store.access.btree.index.B2IRowLockingRR.lockScanRow(B2IRowLockingRR.java:105)
at org.apache.derby.impl.store.access.btree.BTreeForwardScan.fetchRows(BTreeForwardScan.java:305)
at org.apache.derby.impl.store.access.btree.BTreeScan.fetchNextGroup(BTreeScan.java:1585)
at org.apache.derby.impl.sql.execute.BulkTableScanResultSet.reloadArray(BulkTableScanResultSet.java:327)
at org.apache.derby.impl.sql.execute.BulkTableScanResultSet.getNextRowCore(BulkTableScanResultSet.java:282)
at org.apache.derby.impl.sql.execute.BasicNoPutResultSetImpl.getNextRow(BasicNoPutResultSetImpl.java:460)
at org.apache.derby.impl.jdbc.EmbedResultSet.movePosition(EmbedResultSet.java:427)
-  locked org.apache.derby.impl.jdbc.EmbedConnection40@445d374b
at org.apache.derby.impl.jdbc.EmbedResultSet.next(EmbedResultSet.java:371)
       ...

So is using Thread.interrupt() the correct way to interrupt a long running query? Currently our worker threads are using 'java.util.concurrent.ThreadPoolExecutor' to run our queries and it calls 'interrupt()' when it shuts down.

Hi Luke,

Unfortunately, Derby isn't currently able to cope correctly with getting interrupted by Thread.interrupt(). Based on limited information, and as it stands today, I think your options are:
 a) code special shutdown logic for Derby
 b) do not invoke Thread.interrupt()
 c) force Derby to not use NIO (?)

This issue seems like something the Derby community has to address, as it is easily encountered using Java standard libraries (i.e. java.util.concurrent).
It is not clear to me why the lock system fails. Do you have a stack trace from the thread being interrupted?

Also, can you describe your requirements for the process issuing Thread.interrupt()?
I.e, do you want to shut down your application "hard", or just to cancel a single running query?
You may find some helpful information in this thread (see especially the posts from Stephan and Knut): http://old.nabble.com/cancel-a-running-query--td19903183.html


Regards,
--
Kristian



This was with Derby 10.5.3.0 and JRE 1.6.0_19-b04 on Windows Server 2008.


Thanks in advance,


Luke Quinane