db-derby-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kristian Waagan <Kristian.Waa...@Sun.COM>
Subject Re: Hung thread after another thread is interrupted
Date Tue, 22 Jun 2010 08:03:05 GMT
On 22.06.10 08:46, Luke Quinane wrote:
> Hi Kristian,
>
> I've played around with the code a bit during the day and the attached 
> patch seems to fix the problem where another thread is stuck. I've 
> never really look at the Derby code before so I'm not confident that 
> it is correct however it did seem to fix things for me.

Thanks Luke,

I think the person who has worked most on the lock system recently is 
around. Hopefully he'll have a look at your patch.
To me the patch looks okay, but I don't have the cycles to verify it at 
the moment. I will however apply it to trunk and run the regression tests.

My suspicions about write operations getting interrupted were groundless 
in this case, so it seems you have hit a different issue.


Regards,
-- 
Kristian

>
>
> Cheers,
>
> Luke
>
>
> On Tue, Jun 22, 2010 at 11:25 AM, Luke Quinane <luke@nuix.com 
> <mailto:luke@nuix.com>> wrote:
>
>     Hi Kristian,
>
>     I've attached our Derby log file that has all the threads that
>     were interrupted when the problem occurred (quite a few).
>
>     Our application was in the middle of a user initiated query when
>     the user started another operation. For performance reasons we
>     call shutdown() on the previous thread pool which causes the
>     interrupt() call on all the threads. As you can see in the stack
>     traces some threads are querying Lucene and some are querying Derby.
>
>     Mostly we just want to cancel the current query but from the
>     higher levels of the application its not apparent if a thread is
>     processing a Lucene or a Derby query.
>
>
>     Thanks again,
>
>     Luke
>
>
>     On Mon, Jun 21, 2010 at 6:52 PM, Kristian Waagan
>     <Kristian.Waagan@sun.com <mailto:Kristian.Waagan@sun.com>> wrote:
>
>         On 21.06.10 09:20, Luke Quinane wrote:
>
>             Hi all,
>
>             We've seen a problem today where we have several threads
>             querying our database and when one gets interrupted the
>             others are stuck waiting for a lock. Here is the stack
>             trace for the stuck thread(s):
>
>             daemon prio=4
>             "DefaultExecutorService-pool-1-thread-47" Id=98 WAITING on
>             org.apache.derby.impl.services.locks.ActiveLock@6e6f45a1
>             at java.lang.Object.wait(Native Method)
>             -  waiting on
>             org.apache.derby.impl.services.locks.ActiveLock@6e6f45a1
>             at java.lang.Object.wait(Object.java:485)
>             at
>             org.apache.derby.impl.services.locks.ActiveLock.waitForGrant(ActiveLock.java:115)
>             at
>             org.apache.derby.impl.services.locks.ConcurrentLockSet.lockObject(ConcurrentLockSet.java:463)
>             at
>             org.apache.derby.impl.services.locks.ConcurrentLockSet.zeroDurationLockObject(ConcurrentLockSet.java:855)
>             at
>             org.apache.derby.impl.services.locks.AbstractPool.zeroDurationlockObject(AbstractPool.java:297)
>             at
>             org.apache.derby.impl.services.locks.ConcurrentPool.zeroDurationlockObject(ConcurrentPool.java:28)
>             at
>             org.apache.derby.impl.store.raw.xact.RowLocking2nohold.lockRecordForRead(RowLocking2nohold.java:89)
>             at
>             org.apache.derby.impl.store.access.heap.HeapController.lockRow(HeapController.java:520)
>             at
>             org.apache.derby.impl.store.access.heap.HeapController.lockRow(HeapController.java:638)
>             at
>             org.apache.derby.impl.store.access.btree.index.B2IRowLocking3.lockRowOnPage(B2IRowLocking3.java:309)
>             at
>             org.apache.derby.impl.store.access.btree.index.B2IRowLocking3._lockScanRow(B2IRowLocking3.java:599)
>             at
>             org.apache.derby.impl.store.access.btree.index.B2IRowLockingRR.lockScanRow(B2IRowLockingRR.java:105)
>             at
>             org.apache.derby.impl.store.access.btree.BTreeForwardScan.fetchRows(BTreeForwardScan.java:305)
>             at
>             org.apache.derby.impl.store.access.btree.BTreeScan.fetchNextGroup(BTreeScan.java:1585)
>             at
>             org.apache.derby.impl.sql.execute.BulkTableScanResultSet.reloadArray(BulkTableScanResultSet.java:327)
>             at
>             org.apache.derby.impl.sql.execute.BulkTableScanResultSet.getNextRowCore(BulkTableScanResultSet.java:282)
>             at
>             org.apache.derby.impl.sql.execute.BasicNoPutResultSetImpl.getNextRow(BasicNoPutResultSetImpl.java:460)
>             at
>             org.apache.derby.impl.jdbc.EmbedResultSet.movePosition(EmbedResultSet.java:427)
>             -  locked
>             org.apache.derby.impl.jdbc.EmbedConnection40@445d374b
>             at
>             org.apache.derby.impl.jdbc.EmbedResultSet.next(EmbedResultSet.java:371)
>                    ...
>
>             So is using Thread.interrupt() the correct way to
>             interrupt a long running query? Currently our worker
>             threads are using
>             'java.util.concurrent.ThreadPoolExecutor' to run our
>             queries and it calls 'interrupt()' when it shuts down.
>
>
>         Hi Luke,
>
>         Unfortunately, Derby isn't currently able to cope correctly
>         with getting interrupted by Thread.interrupt(). Based on
>         limited information, and as it stands today, I think your
>         options are:
>          a) code special shutdown logic for Derby
>          b) do not invoke Thread.interrupt()
>          c) force Derby to not use NIO (?)
>
>         This issue seems like something the Derby community has to
>         address, as it is easily encountered using Java standard
>         libraries (i.e. java.util.concurrent).
>         It is not clear to me why the lock system fails. Do you have a
>         stack trace from the thread being interrupted?
>
>         Also, can you describe your requirements for the process
>         issuing Thread.interrupt()?
>         I.e, do you want to shut down your application "hard", or just
>         to cancel a single running query?
>         You may find some helpful information in this thread (see
>         especially the posts from Stephan and Knut):
>         http://old.nabble.com/cancel-a-running-query--td19903183.html
>
>
>         Regards,
>         -- 
>         Kristian
>
>
>
>             This was with Derby 10.5.3.0 and JRE 1.6.0_19-b04 on
>             Windows Server 2008.
>
>
>             Thanks in advance,
>
>
>             Luke Quinane
>
>
>
>
>


Mime
View raw message