db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kathey Marsden (Updated) (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (DERBY-5552) Derby threads hanging when using ClientXADataSource and a deadlock or lock timeout occurs
Date Sat, 25 Feb 2012 00:25:49 GMT

     [ https://issues.apache.org/jira/browse/DERBY-5552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Kathey Marsden updated DERBY-5552:
----------------------------------

    Attachment: ReproDerby5552DB2.java
                utilXid.java

Well, I tried this out with DB2 and things are interesting and different.  The first thing
of note is that the default for DB2 is no lock timeout. I had to go into Control Center and
change the database application parameter LOCKTIMEOUT to be something other than -1. 

With DB2 after the lock timeout DB2 doesn't let you use the connection saying 
Exception trying to check conn2 after lock timeout but before explicit rollback
com.ibm.db2.jcc.am.SqlException: [jcc][t4][10342][11669][4.12.55] Application must execute
a rollback. The unit of w
has already been rolled back in the database but other resource managers involved in this
unit of work might not. To
ure integrity of this application, all SQL requests will be rejected until the application
issues a rollback.  ERROR
=-4497, SQLSTATE=null


Then it won't actually let you do a rollback, but will let you do an end.  If you try rollback
it fails with XAER_PROTO but a rollback after end fails with XAER_NOTA if you try to rollback
after end. Statements after the end fail with XA_RBTIMEOUT  so I am not sure actually how
one could use the connection again.

Below is the output and attached is the program I was playing with in case anyone is interested,
but I think the main thing I have learned is that since the spec doesn't really offer any
guidance on this, we just need to do something reasonable and make sure that the global transaction
doesn't get used again except to rollback, which I think the behavior with the patch does
effectively as the activity that happens on the connection after the implicit rollback happens
in a new local transaction.
So I will go ahead and check in the latest patch  with the expanded test.

$ java -Duser=xxxxx  -Dpassword=xxx  ReproDerby5552DB2
Got Expected Lock Timeout Exception DB2 SQL Error: SQLCODE=-911, SQLSTATE=40001, SQLERRMC=68,
DRIVER=4.12.55
Connection ok. got right value
Exception trying to check conn2 after lock timeout but before explicit rollback
com.ibm.db2.jcc.am.SqlException: [jcc][t4][10342][11669][4.12.55] Application must execute
a rollback. The unit of work
has already been rolled back in the database but other resource managers involved in this
unit of work might not. To ens
ure integrity of this application, all SQL requests will be rejected until the application
issues a rollback.  ERRORCODE
=-4497, SQLSTATE=null
        at com.ibm.db2.jcc.am.hd.a(hd.java:660)
        at com.ibm.db2.jcc.am.hd.a(hd.java:60)
        at com.ibm.db2.jcc.am.hd.a(hd.java:75)
        at com.ibm.db2.jcc.am.o.f(o.java:537)
        at com.ibm.db2.jcc.am.o.a(o.java:495)
        at com.ibm.db2.jcc.am.Sqlca.getJDBCMessage(Sqlca.java:334)
        at com.ibm.db2.jcc.am.SqlExceptionContainer.getMessage(SqlExceptionContainer.java:78)
        at com.ibm.db2.jcc.am.SqlTransactionRollbackException.getMessage(SqlTransactionRollbackException.java:52)
        at ReproDerby5552DB2.main(ReproDerby5552DB2.java:61)
Did not get exception on end as with Derby
Got Exception checking conn2 after end
com.ibm.db2.jcc.am.SqlException: [jcc][t4][2041][11392][4.12.55] Error executing XAResource.end().
 Server returned XA_R
BTIMEOUT. ERRORCODE=-4203, SQLSTATE=null
        at com.ibm.db2.jcc.am.hd.a(hd.java:660)
        at com.ibm.db2.jcc.am.hd.a(hd.java:60)
        at com.ibm.db2.jcc.am.hd.a(hd.java:94)
        at com.ibm.db2.jcc.t4.zb.a(zb.java:2755)
        at com.ibm.db2.jcc.t4.c.Xb(c.java:271)
        at com.ibm.db2.jcc.am.o.g(o.java:340)
        at com.ibm.db2.jcc.t4.a.g(a.java:631)
        at com.ibm.db2.jcc.am.o.a(o.java:214)
        at com.ibm.db2.jcc.am.mn.a(mn.java:3073)
        at com.ibm.db2.jcc.am.mn.a(mn.java:686)
        at com.ibm.db2.jcc.am.mn.executeQuery(mn.java:670)
        at ReproDerby5552DB2.checkConn(ReproDerby5552DB2.java:106)
        at ReproDerby5552DB2.main(ReproDerby5552DB2.java:86)
Rolling back xid2
Exception in thread "main" com.ibm.db2.jcc.am.XaException: [jcc][t4][2041][12326][4.12.55]
Error executing XAResource.ro
llback().  Server returned XAER_NOTA. ERRORCODE=-4203, SQLSTATE=null
        at com.ibm.db2.jcc.am.hd.c(hd.java:453)
        at com.ibm.db2.jcc.t4.zb.b(zb.java:2773)
        at com.ibm.db2.jcc.t4.ac.b(ac.java:1546)
        at com.ibm.db2.jcc.t4.ac.a(ac.java:1326)
        at com.ibm.db2.jcc.t4.ac.a(ac.java:1321)
        at com.ibm.db2.jcc.t4.ac.rollback(ac.java:1310)
        at ReproDerby5552DB2.main(ReproDerby5552DB2.java:94)

kmarsden@IBM-JDPM42DBIO2 ~/repro/derby-5552
$
















































































$ java ReproDerby5552DB2
Got Expected Lock Timeout Exception DB2 SQL Error: SQLCODE=-911, SQLSTATE=40001, SQLERRMC=6
Connection ok. got right value
Exception trying to check conn2 after lock timeout but before explicit rollback
com.ibm.db2.jcc.am.SqlException: [jcc][t4][10342][11669][4.12.55] Application must execute
has already been rolled back in the database but other resource managers involved in this
u
ure integrity of this application, all SQL requests will be rejected until the application
=-4497, SQLSTATE=null
        at com.ibm.db2.jcc.am.hd.a(hd.java:660)
        at com.ibm.db2.jcc.am.hd.a(hd.java:60)
        at com.ibm.db2.jcc.am.hd.a(hd.java:75)
        at com.ibm.db2.jcc.am.o.f(o.java:537)
        at com.ibm.db2.jcc.am.o.a(o.java:495)
        at com.ibm.db2.jcc.am.Sqlca.getJDBCMessage(Sqlca.java:334)
        at com.ibm.db2.jcc.am.SqlExceptionContainer.getMessage(SqlExceptionContainer.java:7
        at com.ibm.db2.jcc.am.SqlTransactionRollbackException.getMessage(SqlTransactionRoll
        at ReproDerby5552DB2.main(ReproDerby5552DB2.java:61)
Did not get exception on end as with Derby
Rolling back xid2
Exception in thread "main" com.ibm.db2.jcc.am.XaException: [jcc][t4][2041][12326][4.12.55]
llback().  Server returned XAER_NOTA. ERRORCODE=-4203, SQLSTATE=null
        at com.ibm.db2.jcc.am.hd.c(hd.java:453)
        at com.ibm.db2.jcc.t4.zb.b(zb.java:2773)
        at com.ibm.db2.jcc.t4.ac.b(ac.java:1546)
        at com.ibm.db2.jcc.t4.ac.a(ac.java:1326)
        at com.ibm.db2.jcc.t4.ac.a(ac.java:1321)
        at com.ibm.db2.jcc.t4.ac.rollback(ac.java:1310)
        at ReproDerby5552DB2.main(ReproDerby5552DB2.java:86)
Caused by: com.ibm.db2.jcc.am.XaException: [jcc][t4][2041][12326][4.12.55] Error executing
urned XA_RBTIMEOUT. ERRORCODE=-4203, SQLSTATE=null
        at com.ibm.db2.jcc.am.hd.c(hd.java:453)
        at com.ibm.db2.jcc.t4.zb.b(zb.java:2773)
        at com.ibm.db2.jcc.t4.ac.b(ac.java:1520)
        ... 4 more

kmarsden@IBM-JDPM42DBIO2 ~/repro/derby-5552
$






                
> Derby threads hanging when using ClientXADataSource and a deadlock or lock timeout occurs
> -----------------------------------------------------------------------------------------
>
>                 Key: DERBY-5552
>                 URL: https://issues.apache.org/jira/browse/DERBY-5552
>             Project: Derby
>          Issue Type: Bug
>          Components: Network Server
>    Affects Versions: 10.8.1.2
>         Environment: Solaris 10, Glassfish V2.1.1,
>            Reporter: Brett Bergquist
>            Assignee: Kathey Marsden
>            Priority: Blocker
>              Labels: derby_triage10_9
>         Attachments: DERBY-5552-p1.patch, DERBY-5552-p2.patch, ReproDerby5552DB2.java,
ReproDerby5552LockTimeout.java, appserverstack.txt, client.tar.Z, derby-5552_withexpanded_test_diff.txt,
derby-5552_withtest_diff.txt, derby-5552_withtest_diff.txt, derby.log, derbystackatshutdown.txt,
execute.patch, transactionsleft.txt, utilXid.java
>
>
> The issue arrives when multiple XA transactions are done in parallel and there is either
a lock timeout or a lock deadlock detected.  When this happens the connection is leaked in
the Glassfish connection pool and the client thread hangs in "org.apache.derby.client.netReply.fill(Reply.java:172)".
 
> Shutting down the app server fails because the thread has a lock in "org.apache.derby.client.net.NetConnection40"
and another task is calling "org.apache.derby.client.ClientPooledConnection.close(ClientPooledConnection.java:214)"
which is waiting for the lock.
> Killing the appsever using "kill" and then attempting to shutdown Derby network server
causes the Network Server to hang.  One of the threads hangs waiting for a lock at "org.apache.derby.impl.drda.NeworkServerControlImpl.removeFromSessionTable(NetworkServerControlImpl.java:1525)"
and the "main" thread has this locked at "org.apache.derby.impl.drda.NetworkServerControlImpl.executeWork(NetworkServerControlImpl.java:2242)"
and it itself is waiting for a lock which belongs to a thread that is stuck at "org.apache.derby.impl.services.locks.ActiveLock.waitForGrant(ActiveLock.java:118)
which is in the TIMED_WAITING state.
> Only by killing the Network Server using "kill" is possible at this point.
> There are transactions left even though all clients have been removed.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message