I'm encountering a lock-up in Derby when I put some load (minor) on our server.  However, I'm not sure if it's a DB deadlock or if it's a thread-lock in Derby's internals.

Unfortunately, I don't have a simple recreation test-case at this time.  Below is a threaddump of active Derby threads at the time of the lock-up.  However, first some background -- which maybe either muddy the waters or clarify things to developers.

I am running Derby 10.6.1.0 and see this issue, but one of our customers saw this issue on 10.5.3.0.  I am using Bitronix JTA for XA transactions.  While I do not think the problem is Bitronix, one tuning parameter in Bitronix seem to have an effect on the bug (i.e. tickle the lock-up in Derby).

The pseudo code being run looks like this:

{
   1. Start transaction

   2. Create connection

   3. SELECT columns FROM table WHERE something=?

   4. Close connection

   5. Create connection

   6. DELETE FROM table WHERE something_id=?    // note: deletes same row that was queried above

   7. Close connection

   8. End transaction
}

This code is being run by multiple threads.  While logically sub-optimal, for example the 2 connections, it is not something I control because of Hibernate between me and the DB.

Now, the Bitronix configuration:

a. There is a connection pool.
b. PreparedStatements are cached (per connection)

As mentioned above, one tuning parameter in Bitronix affects whether this bug is tickled.  If I turn off the PreparedStatement cache, the lock-up does not occur.

So, finally, here is the thread dump containing the active Derby related-threads:

"Jetty-9" daemon prio=5 tid=101e72800 nid=0x13530c000 runnable [135309000]
   java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at org.apache.derby.client.net.Reply.fill(Unknown Source)
at org.apache.derby.client.net.Reply.ensureALayerDataInBuffer(Unknown Source)
at org.apache.derby.client.net.Reply.readDssHeader(Unknown Source)
at org.apache.derby.client.net.Reply.startSameIdChainParse(Unknown Source)
at org.apache.derby.client.net.NetStatementReply.readExecute(Unknown Source)
at org.apache.derby.client.net.StatementReply.readExecute(Unknown Source)
at org.apache.derby.client.net.NetPreparedStatement.readExecute_(Unknown Source)
at org.apache.derby.client.am.PreparedStatement.readExecute(Unknown Source)
at org.apache.derby.client.am.PreparedStatement.executeBatchRequestX(Unknown Source)
at org.apache.derby.client.am.PreparedStatement.executeBatchX(Unknown Source)
- locked <110a6a610> (a org.apache.derby.client.net.NetConnection40)
at org.apache.derby.client.am.PreparedStatement.executeBatch(Unknown Source)
- locked <110a6a610> (a org.apache.derby.client.net.NetConnection40)
at sun.reflect.GeneratedMethodAccessor77.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at bitronix.tm.resource.jdbc.BaseProxyHandlerClass.invoke(BaseProxyHandlerClass.java:41)
at $Proxy47.executeBatch(Unknown Source)
at org.hibernate.jdbc.BatchingBatcher.doExecuteBatch(BatchingBatcher.java:70)
at org.hibernate.jdbc.AbstractBatcher.executeBatch(AbstractBatcher.java:268)
at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:263)
at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:184)
at org.hibernate.event.def.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:321)
at org.hibernate.event.def.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:51)
at org.hibernate.impl.SessionImpl.flush(SessionImpl.java:1206)
at org.hibernate.impl.SessionImpl.managedFlush(SessionImpl.java:375)
at org.hibernate.transaction.CacheSynchronization.beforeCompletion(CacheSynchronization.java:88)
at bitronix.tm.BitronixTransaction.fireBeforeCompletionEvent(BitronixTransaction.java:398)
at bitronix.tm.BitronixTransaction.commit(BitronixTransaction.java:143)
at bitronix.tm.BitronixTransactionManager.commit(BitronixTransactionManager.java:103)
at org.ziptie.zap.jta.TransactionElf.commit(TransactionElf.java:68)
at org.ziptie.server.web.ZTransactionFilter.doFilter(ZTransactionFilter.java:86)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at org.ziptie.server.security.ZSecurityFilter.doFilter(ZSecurityFilter.java:60)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at org.ziptie.zap.metro.ZThreadContextFilter.doFilter(ZThreadContextFilter.java:49)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388)
at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
at org.ziptie.zap.web.internal.ZContext.handle(ZContext.java:148)
at org.ziptie.zap.web.ZSessionHandler.handle(ZSessionHandler.java:108)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:938)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:755)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
at org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:451)

"DRDAConnThread_40" daemon prio=5 tid=132121800 nid=0x13555a000 in Object.wait() [135558000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <125674608> (a org.apache.derby.impl.services.locks.ActiveLock)
at org.apache.derby.impl.services.locks.ActiveLock.waitForGrant(Unknown Source)
- locked <125674608> (a org.apache.derby.impl.services.locks.ActiveLock)
at org.apache.derby.impl.services.locks.ConcurrentLockSet.lockObject(Unknown Source)
at org.apache.derby.impl.services.locks.AbstractPool.lockObject(Unknown Source)
at org.apache.derby.impl.services.locks.ConcurrentPool.lockObject(Unknown Source)
at org.apache.derby.impl.store.raw.xact.ContainerLocking3.lockContainer(Unknown Source)
at org.apache.derby.impl.store.raw.data.BaseContainerHandle.useContainer(Unknown Source)
at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.openContainer(Unknown Source)
at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.openContainer(Unknown Source)
at org.apache.derby.impl.store.raw.xact.Xact.openContainer(Unknown Source)
at org.apache.derby.impl.store.access.conglomerate.OpenConglomerate.init(Unknown Source)
at org.apache.derby.impl.store.access.heap.Heap.open(Unknown Source)
at org.apache.derby.impl.store.access.RAMTransaction.openConglomerate(Unknown Source)
at org.apache.derby.impl.store.access.RAMTransaction.openCompiledConglomerate(Unknown Source)
at org.apache.derby.impl.sql.execute.DependentResultSet.openHeapConglomerateController(Unknown Source)
at org.apache.derby.impl.sql.execute.DependentResultSet.openCore(Unknown Source)
at org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.openCore(Unknown Source)
at org.apache.derby.impl.sql.execute.DeleteResultSet.setup(Unknown Source)
at org.apache.derby.impl.sql.execute.DeleteCascadeResultSet.setup(Unknown Source)
at org.apache.derby.impl.sql.execute.DeleteCascadeResultSet.setup(Unknown Source)
at org.apache.derby.impl.sql.execute.DeleteCascadeResultSet.open(Unknown Source)
at org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(Unknown Source)
at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source)
- locked <1250fa928> (a org.apache.derby.impl.jdbc.EmbedConnection40)
at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedPreparedStatement.execute(Unknown Source)
at org.apache.derby.iapi.jdbc.BrokeredPreparedStatement.execute(Unknown Source)
at org.apache.derby.impl.drda.DRDAStatement.execute(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.parseEXCSQLSTTobjects(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.parseEXCSQLSTT(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)

"DRDAConnThread_39" daemon prio=5 tid=133820800 nid=0x131f2f000 in Object.wait() [131f2e000]
   java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <1109ce228> (a org.apache.derby.impl.sql.GenericPreparedStatement)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.sql.GenericStatement.prepMinion(Unknown Source)
- locked <1109ce228> (a org.apache.derby.impl.sql.GenericPreparedStatement)
at org.apache.derby.impl.sql.GenericStatement.prepare(Unknown Source)
at org.apache.derby.impl.sql.GenericPreparedStatement.rePrepare(Unknown Source)
at org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(Unknown Source)
at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source)
- locked <124ea5630> (a org.apache.derby.impl.jdbc.EmbedConnection40)
at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedPreparedStatement.execute(Unknown Source)
at org.apache.derby.iapi.jdbc.BrokeredPreparedStatement.execute(Unknown Source)
at org.apache.derby.impl.drda.DRDAStatement.execute(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.parseEXCSQLSTTobjects(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.parseEXCSQLSTT(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)

"DRDAConnThread_38" daemon prio=5 tid=101ab6000 nid=0x12f78d000 in Object.wait() [12f78b000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <1256ab2e8> (a org.apache.derby.impl.services.locks.ActiveLock)
at org.apache.derby.impl.services.locks.ActiveLock.waitForGrant(Unknown Source)
- locked <1256ab2e8> (a org.apache.derby.impl.services.locks.ActiveLock)
at org.apache.derby.impl.services.locks.ConcurrentLockSet.lockObject(Unknown Source)
at org.apache.derby.impl.services.locks.AbstractPool.lockObject(Unknown Source)
at org.apache.derby.impl.services.locks.ConcurrentPool.lockObject(Unknown Source)
at org.apache.derby.impl.store.raw.xact.ContainerLocking3.lockContainer(Unknown Source)
at org.apache.derby.impl.store.raw.data.BaseContainerHandle.useContainer(Unknown Source)
at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.openContainer(Unknown Source)
at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.openContainer(Unknown Source)
at org.apache.derby.impl.store.raw.xact.Xact.openContainer(Unknown Source)
at org.apache.derby.impl.store.access.conglomerate.OpenConglomerate.init(Unknown Source)
at org.apache.derby.impl.store.access.heap.Heap.open(Unknown Source)
at org.apache.derby.impl.store.access.RAMTransaction.openConglomerate(Unknown Source)
at org.apache.derby.impl.store.access.RAMTransaction.openCompiledConglomerate(Unknown Source)
at org.apache.derby.impl.sql.execute.DependentResultSet.openHeapConglomerateController(Unknown Source)
at org.apache.derby.impl.sql.execute.DependentResultSet.openCore(Unknown Source)
at org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.openCore(Unknown Source)
at org.apache.derby.impl.sql.execute.DeleteResultSet.setup(Unknown Source)
at org.apache.derby.impl.sql.execute.DeleteCascadeResultSet.setup(Unknown Source)
at org.apache.derby.impl.sql.execute.DeleteCascadeResultSet.setup(Unknown Source)
at org.apache.derby.impl.sql.execute.DeleteCascadeResultSet.open(Unknown Source)
at org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(Unknown Source)
at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source)
- locked <125060ff0> (a org.apache.derby.impl.jdbc.EmbedConnection40)
at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedPreparedStatement.execute(Unknown Source)
at org.apache.derby.iapi.jdbc.BrokeredPreparedStatement.execute(Unknown Source)
at org.apache.derby.impl.drda.DRDAStatement.execute(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.parseEXCSQLSTTobjects(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.parseEXCSQLSTT(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)

"DRDAConnThread_37" daemon prio=5 tid=133890800 nid=0x1373f7000 in Object.wait() [1373f5000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <12573fe10> (a org.apache.derby.impl.services.locks.ActiveLock)
at org.apache.derby.impl.services.locks.ActiveLock.waitForGrant(Unknown Source)
- locked <12573fe10> (a org.apache.derby.impl.services.locks.ActiveLock)
at org.apache.derby.impl.services.locks.ConcurrentLockSet.lockObject(Unknown Source)
at org.apache.derby.impl.services.locks.AbstractPool.lockObject(Unknown Source)
at org.apache.derby.impl.services.locks.ConcurrentPool.lockObject(Unknown Source)
at org.apache.derby.impl.store.raw.xact.RowLocking2.lockContainer(Unknown Source)
at org.apache.derby.impl.store.raw.data.BaseContainerHandle.useContainer(Unknown Source)
at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.openContainer(Unknown Source)
at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.openContainer(Unknown Source)
at org.apache.derby.impl.store.raw.xact.Xact.openContainer(Unknown Source)
at org.apache.derby.impl.store.access.conglomerate.OpenConglomerate.init(Unknown Source)
at org.apache.derby.impl.store.access.heap.Heap.open(Unknown Source)
at org.apache.derby.impl.store.access.RAMTransaction.openConglomerate(Unknown Source)
at org.apache.derby.impl.store.access.RAMTransaction.openConglomerate(Unknown Source)
at org.apache.derby.impl.sql.catalog.DataDictionaryImpl.getRowLocationTemplate(Unknown Source)
at org.apache.derby.impl.sql.compile.FromBaseTable.generateRefActionDependentTableScan(Unknown Source)
at org.apache.derby.impl.sql.compile.FromBaseTable.generateResultSet(Unknown Source)
at org.apache.derby.impl.sql.compile.FromBaseTable.generate(Unknown Source)
at org.apache.derby.impl.sql.compile.ProjectRestrictNode.generateMinion(Unknown Source)
at org.apache.derby.impl.sql.compile.ProjectRestrictNode.generate(Unknown Source)
at org.apache.derby.impl.sql.compile.ProjectRestrictNode.generateMinion(Unknown Source)
at org.apache.derby.impl.sql.compile.ProjectRestrictNode.generate(Unknown Source)
at org.apache.derby.impl.sql.compile.DeleteNode.generate(Unknown Source)
at org.apache.derby.impl.sql.compile.DeleteNode.generate(Unknown Source)
at org.apache.derby.impl.sql.compile.StatementNode.generate(Unknown Source)
at org.apache.derby.impl.sql.GenericStatement.prepMinion(Unknown Source)
at org.apache.derby.impl.sql.GenericStatement.prepare(Unknown Source)
at org.apache.derby.impl.sql.GenericPreparedStatement.rePrepare(Unknown Source)
at org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(Unknown Source)
at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source)
- locked <124f97c40> (a org.apache.derby.impl.jdbc.EmbedConnection40)
at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedPreparedStatement.execute(Unknown Source)
at org.apache.derby.iapi.jdbc.BrokeredPreparedStatement.execute(Unknown Source)
at org.apache.derby.impl.drda.DRDAStatement.execute(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.parseEXCSQLSTTobjects(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.parseEXCSQLSTT(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)

"Jetty-8" daemon prio=5 tid=132020800 nid=0x13731b000 runnable [137318000]
   java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at org.apache.derby.client.net.Reply.fill(Unknown Source)
at org.apache.derby.client.net.Reply.ensureALayerDataInBuffer(Unknown Source)
at org.apache.derby.client.net.Reply.readDssHeader(Unknown Source)
at org.apache.derby.client.net.Reply.startSameIdChainParse(Unknown Source)
at org.apache.derby.client.net.NetStatementReply.readExecute(Unknown Source)
at org.apache.derby.client.net.StatementReply.readExecute(Unknown Source)
at org.apache.derby.client.net.NetPreparedStatement.readExecute_(Unknown Source)
at org.apache.derby.client.am.PreparedStatement.readExecute(Unknown Source)
at org.apache.derby.client.am.PreparedStatement.executeBatchRequestX(Unknown Source)
at org.apache.derby.client.am.PreparedStatement.executeBatchX(Unknown Source)
- locked <1109fff30> (a org.apache.derby.client.net.NetConnection40)
at org.apache.derby.client.am.PreparedStatement.executeBatch(Unknown Source)
- locked <1109fff30> (a org.apache.derby.client.net.NetConnection40)
at sun.reflect.GeneratedMethodAccessor77.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at bitronix.tm.resource.jdbc.BaseProxyHandlerClass.invoke(BaseProxyHandlerClass.java:41)
at $Proxy47.executeBatch(Unknown Source)
at org.hibernate.jdbc.BatchingBatcher.doExecuteBatch(BatchingBatcher.java:70)
at org.hibernate.jdbc.AbstractBatcher.executeBatch(AbstractBatcher.java:268)
at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:263)
at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:184)
at org.hibernate.event.def.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:321)
at org.hibernate.event.def.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:51)
at org.hibernate.impl.SessionImpl.flush(SessionImpl.java:1206)
at org.hibernate.impl.SessionImpl.managedFlush(SessionImpl.java:375)
at org.hibernate.transaction.CacheSynchronization.beforeCompletion(CacheSynchronization.java:88)
at bitronix.tm.BitronixTransaction.fireBeforeCompletionEvent(BitronixTransaction.java:398)
at bitronix.tm.BitronixTransaction.commit(BitronixTransaction.java:143)
at bitronix.tm.BitronixTransactionManager.commit(BitronixTransactionManager.java:103)
at org.ziptie.zap.jta.TransactionElf.commit(TransactionElf.java:68)
at org.ziptie.server.web.ZTransactionFilter.doFilter(ZTransactionFilter.java:86)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at org.ziptie.server.security.ZSecurityFilter.doFilter(ZSecurityFilter.java:60)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at org.ziptie.zap.metro.ZThreadContextFilter.doFilter(ZThreadContextFilter.java:49)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388)
at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
at org.ziptie.zap.web.internal.ZContext.handle(ZContext.java:148)
at org.ziptie.zap.web.ZSessionHandler.handle(ZSessionHandler.java:108)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:938)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:755)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
at org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:451)

"Jetty-6" daemon prio=5 tid=134884800 nid=0x12f7bd000 runnable [12f7ba000]
   java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at org.apache.derby.client.net.Reply.fill(Unknown Source)
at org.apache.derby.client.net.Reply.ensureALayerDataInBuffer(Unknown Source)
at org.apache.derby.client.net.Reply.readDssHeader(Unknown Source)
at org.apache.derby.client.net.Reply.startSameIdChainParse(Unknown Source)
at org.apache.derby.client.net.NetStatementReply.readExecute(Unknown Source)
at org.apache.derby.client.net.StatementReply.readExecute(Unknown Source)
at org.apache.derby.client.net.NetPreparedStatement.readExecute_(Unknown Source)
at org.apache.derby.client.am.PreparedStatement.readExecute(Unknown Source)
at org.apache.derby.client.am.PreparedStatement.executeBatchRequestX(Unknown Source)
at org.apache.derby.client.am.PreparedStatement.executeBatchX(Unknown Source)
- locked <110a27498> (a org.apache.derby.client.net.NetConnection40)
at org.apache.derby.client.am.PreparedStatement.executeBatch(Unknown Source)
- locked <110a27498> (a org.apache.derby.client.net.NetConnection40)
at sun.reflect.GeneratedMethodAccessor77.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at bitronix.tm.resource.jdbc.BaseProxyHandlerClass.invoke(BaseProxyHandlerClass.java:41)
at $Proxy47.executeBatch(Unknown Source)
at org.hibernate.jdbc.BatchingBatcher.doExecuteBatch(BatchingBatcher.java:70)
at org.hibernate.jdbc.AbstractBatcher.executeBatch(AbstractBatcher.java:268)
at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:263)
at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:184)
at org.hibernate.event.def.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:321)
at org.hibernate.event.def.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:51)
at org.hibernate.impl.SessionImpl.flush(SessionImpl.java:1206)
at org.hibernate.impl.SessionImpl.managedFlush(SessionImpl.java:375)
at org.hibernate.transaction.CacheSynchronization.beforeCompletion(CacheSynchronization.java:88)
at bitronix.tm.BitronixTransaction.fireBeforeCompletionEvent(BitronixTransaction.java:398)
at bitronix.tm.BitronixTransaction.commit(BitronixTransaction.java:143)
at bitronix.tm.BitronixTransactionManager.commit(BitronixTransactionManager.java:103)
at org.ziptie.zap.jta.TransactionElf.commit(TransactionElf.java:68)
at org.ziptie.server.web.ZTransactionFilter.doFilter(ZTransactionFilter.java:86)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at org.ziptie.server.security.ZSecurityFilter.doFilter(ZSecurityFilter.java:60)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at org.ziptie.zap.metro.ZThreadContextFilter.doFilter(ZThreadContextFilter.java:49)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388)
at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
at org.ziptie.zap.web.internal.ZContext.handle(ZContext.java:148)
at org.ziptie.zap.web.ZSessionHandler.handle(ZSessionHandler.java:108)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:938)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:755)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
at org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:451)

"DRDAConnThread_29" daemon prio=5 tid=103216000 nid=0x1372b2000 runnable [1372b1000]
   java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at org.apache.derby.impl.drda.DDMReader.fill(Unknown Source)
at org.apache.derby.impl.drda.DDMReader.ensureALayerDataInBuffer(Unknown Source)
at org.apache.derby.impl.drda.DDMReader.readDssHeader(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)

"DRDAConnThread_28" daemon prio=5 tid=1018f5800 nid=0x13697b000 runnable [13697a000]
   java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at org.apache.derby.impl.drda.DDMReader.fill(Unknown Source)
at org.apache.derby.impl.drda.DDMReader.ensureALayerDataInBuffer(Unknown Source)
at org.apache.derby.impl.drda.DDMReader.readDssHeader(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)

"DRDAConnThread_26" daemon prio=5 tid=130304000 nid=0x136821000 in Object.wait() [13681f000]
   java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <1109ce228> (a org.apache.derby.impl.sql.GenericPreparedStatement)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.sql.GenericStatement.prepMinion(Unknown Source)
- locked <1109ce228> (a org.apache.derby.impl.sql.GenericPreparedStatement)
at org.apache.derby.impl.sql.GenericStatement.prepare(Unknown Source)
at org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.prepareInternalStatement(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedPreparedStatement.<init>(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedPreparedStatement20.<init>(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedPreparedStatement30.<init>(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedPreparedStatement40.<init>(Unknown Source)
at org.apache.derby.jdbc.Driver40.newEmbedPreparedStatement(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedConnection.prepareStatement(Unknown Source)
- locked <12560bf10> (a org.apache.derby.impl.jdbc.EmbedConnection40)
at org.apache.derby.impl.jdbc.EmbedConnection.prepareStatement(Unknown Source)
at org.apache.derby.iapi.jdbc.BrokeredPreparedStatement30.createDuplicateStatement(Unknown Source)
at org.apache.derby.jdbc.XAStatementControl.getRealPreparedStatement(Unknown Source)
at org.apache.derby.iapi.jdbc.BrokeredPreparedStatement.getPreparedStatement(Unknown Source)
at org.apache.derby.iapi.jdbc.BrokeredPreparedStatement.setInt(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.readAndSetParams(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.parseSQLDTA_work(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.parseSQLDTA(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.parseEXCSQLSTTobjects(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.parseEXCSQLSTT(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)

"Jetty-4" daemon prio=5 tid=130654800 nid=0x13575c000 runnable [135759000]
   java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at org.apache.derby.client.net.Reply.fill(Unknown Source)
at org.apache.derby.client.net.Reply.ensureALayerDataInBuffer(Unknown Source)
at org.apache.derby.client.net.Reply.readDssHeader(Unknown Source)
at org.apache.derby.client.net.Reply.startSameIdChainParse(Unknown Source)
at org.apache.derby.client.net.NetStatementReply.readExecute(Unknown Source)
at org.apache.derby.client.net.StatementReply.readExecute(Unknown Source)
at org.apache.derby.client.net.NetPreparedStatement.readExecute_(Unknown Source)
at org.apache.derby.client.am.PreparedStatement.readExecute(Unknown Source)
at org.apache.derby.client.am.PreparedStatement.executeBatchRequestX(Unknown Source)
at org.apache.derby.client.am.PreparedStatement.executeBatchX(Unknown Source)
- locked <1103d7120> (a org.apache.derby.client.net.NetConnection40)
at org.apache.derby.client.am.PreparedStatement.executeBatch(Unknown Source)
- locked <1103d7120> (a org.apache.derby.client.net.NetConnection40)
at sun.reflect.GeneratedMethodAccessor77.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at bitronix.tm.resource.jdbc.BaseProxyHandlerClass.invoke(BaseProxyHandlerClass.java:41)
at $Proxy47.executeBatch(Unknown Source)
at org.hibernate.jdbc.BatchingBatcher.doExecuteBatch(BatchingBatcher.java:70)
at org.hibernate.jdbc.AbstractBatcher.executeBatch(AbstractBatcher.java:268)
at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:263)
at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:184)
at org.hibernate.event.def.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:321)
at org.hibernate.event.def.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:51)
at org.hibernate.impl.SessionImpl.flush(SessionImpl.java:1206)
at org.hibernate.impl.SessionImpl.managedFlush(SessionImpl.java:375)
at org.hibernate.transaction.CacheSynchronization.beforeCompletion(CacheSynchronization.java:88)
at bitronix.tm.BitronixTransaction.fireBeforeCompletionEvent(BitronixTransaction.java:398)
at bitronix.tm.BitronixTransaction.commit(BitronixTransaction.java:143)
at bitronix.tm.BitronixTransactionManager.commit(BitronixTransactionManager.java:103)
at org.ziptie.zap.jta.TransactionElf.commit(TransactionElf.java:68)
at org.ziptie.server.web.ZTransactionFilter.doFilter(ZTransactionFilter.java:86)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at org.ziptie.server.security.ZSecurityFilter.doFilter(ZSecurityFilter.java:60)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at org.ziptie.zap.metro.ZThreadContextFilter.doFilter(ZThreadContextFilter.java:49)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388)
at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
at org.ziptie.zap.web.internal.ZContext.handle(ZContext.java:148)
at org.ziptie.zap.web.ZSessionHandler.handle(ZSessionHandler.java:108)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:938)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:755)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
at org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:451)

"Jetty-1" daemon prio=5 tid=101e7a800 nid=0x1353ed000 runnable [1353ea000]
   java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at org.apache.derby.client.net.Reply.fill(Unknown Source)
at org.apache.derby.client.net.Reply.ensureALayerDataInBuffer(Unknown Source)
at org.apache.derby.client.net.Reply.readDssHeader(Unknown Source)
at org.apache.derby.client.net.Reply.startSameIdChainParse(Unknown Source)
at org.apache.derby.client.net.NetStatementReply.readExecute(Unknown Source)
at org.apache.derby.client.net.StatementReply.readExecute(Unknown Source)
at org.apache.derby.client.net.NetPreparedStatement.readExecute_(Unknown Source)
at org.apache.derby.client.am.PreparedStatement.readExecute(Unknown Source)
at org.apache.derby.client.am.PreparedStatement.executeBatchRequestX(Unknown Source)
at org.apache.derby.client.am.PreparedStatement.executeBatchX(Unknown Source)
- locked <11065f0f8> (a org.apache.derby.client.net.NetConnection40)
at org.apache.derby.client.am.PreparedStatement.executeBatch(Unknown Source)
- locked <11065f0f8> (a org.apache.derby.client.net.NetConnection40)
at sun.reflect.GeneratedMethodAccessor77.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at bitronix.tm.resource.jdbc.BaseProxyHandlerClass.invoke(BaseProxyHandlerClass.java:41)
at $Proxy47.executeBatch(Unknown Source)
at org.hibernate.jdbc.BatchingBatcher.doExecuteBatch(BatchingBatcher.java:70)
at org.hibernate.jdbc.AbstractBatcher.executeBatch(AbstractBatcher.java:268)
at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:263)
at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:184)
at org.hibernate.event.def.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:321)
at org.hibernate.event.def.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:51)
at org.hibernate.impl.SessionImpl.flush(SessionImpl.java:1206)
at org.hibernate.impl.SessionImpl.managedFlush(SessionImpl.java:375)
at org.hibernate.transaction.CacheSynchronization.beforeCompletion(CacheSynchronization.java:88)
at bitronix.tm.BitronixTransaction.fireBeforeCompletionEvent(BitronixTransaction.java:398)
at bitronix.tm.BitronixTransaction.commit(BitronixTransaction.java:143)
at bitronix.tm.BitronixTransactionManager.commit(BitronixTransactionManager.java:103)
at org.ziptie.zap.jta.TransactionElf.commit(TransactionElf.java:68)
at org.ziptie.server.web.ZTransactionFilter.doFilter(ZTransactionFilter.java:86)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at org.ziptie.server.security.ZSecurityFilter.doFilter(ZSecurityFilter.java:60)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at org.ziptie.zap.metro.ZThreadContextFilter.doFilter(ZThreadContextFilter.java:49)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388)
at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
at org.ziptie.zap.web.internal.ZContext.handle(ZContext.java:148)
at org.ziptie.zap.web.ZSessionHandler.handle(ZSessionHandler.java:108)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:938)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:755)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
at org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:451)

"DRDAConnThread_7" daemon prio=5 tid=101848000 nid=0x135267000 runnable [135266000]
   java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at org.apache.derby.impl.drda.DDMReader.fill(Unknown Source)
at org.apache.derby.impl.drda.DDMReader.ensureALayerDataInBuffer(Unknown Source)
at org.apache.derby.impl.drda.DDMReader.readDssHeader(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)

From looking at this, it seems the "lock-up" is on the server-side of the conversion.  The client threads (named Jetty-*) are basically all in socketRead() waiting for a response from the server.

Any thoughts/help would be appreciated.  I will create a bug in Jira if I get a nod from a developer that it looks like a real issue.

Thanks,
Brett