I cannot upgrade to 10.8.8.2 at this point as this is in production without upgrade possibility.  A also at this point, I don’t want to change to many variables and have the problem disappear.  Right now I can get it to reproduce pretty readily.

 

I was able to connect with a debug and indeed the thread is blocking on the “java.net.SocketInputStream.read”.  Through the debugger if I force an interrupt, I returns from the socket call with an I/O error.  So does this mean that the problem is in the derby server side since the socket is  valid and not closed before I issue the interrupt?

 

I am going to enable tracing and see what I find.

 

From: Katherine Marsden [mailto:kmarsdenderby@sbcglobal.net]
Sent: Wednesday, December 21, 2011 11:14 AM
To: derby-dev@db.apache.org
Subject: Re: Problem with a deadlock with Derby 10.8.1.2 and Glassfish V2.1.1

 

On 12/21/2011 6:14 AM, Bergquist, Brett wrote:

I have a problem with a deadlock with Derby 10.8.1.2 and Glassfish V2.1.1 but at this point I cannot tell if it is a Derby problem or a Glassfish problem so I am looking for some guidance. 

 

A database pool is created using the “org.apache.derby.jdbc.ClientXADataSource” XADataSource.   Connection validation is turned on and uses “auto-commit” to validate the connection and attempts to close all connections on any validation failure.

 

The deadlock is triggered when I have a couple of transactions that cause a Derby locks deadlock.  Deadlock detection and tracing is turned on in Derby and it correctly detects the locking deadlock and terminates one of the transactions.   Soon after this, however,  Glassfish becomes stuck.  Here is the only reference to Derby in the “jstack” output of Glassfish:

 

"PMCollectionActivationManager.newPmTestResult" daemon prio=3 tid=0x0000000103926800 nid=0xaf runnable [0xfffffffc218fa000]

   java.lang.Thread.State: RUNNABLE

                at java.net.SocketInputStream.socketRead0(Native Method)

                at java.net.SocketInputStream.read(SocketInputStream.java:129)

                at org.apache.derby.client.net.Reply.fill(Unknown Source)

                at org.apache.derby.client.net.Reply.ensureALayerDataInBuffer(Unknown Source)

                at org.apache.derby.client.net.Reply.readDssHeader(Unknown Source)

                at org.apache.derby.client.net.Reply.startSameIdChainParse(Unknown Source)

                at org.apache.derby.client.net.NetXAConnectionReply.readLocalXACommit(Unknown Source)

                at org.apache.derby.client.net.NetXAConnection.readLocalXACommit_(Unknown Source)

                at org.apache.derby.client.net.NetXAConnection.readCommit(Unknown Source)

                at org.apache.derby.client.net.NetConnection.readXACommit_(Unknown Source)

                at org.apache.derby.client.am.Connection.readCommit(Unknown Source)

                at org.apache.derby.client.am.Connection.readAutoCommit(Unknown Source)

                at org.apache.derby.client.am.Connection.flowClose(Unknown Source)

                at org.apache.derby.client.am.Connection.closeResourcesX(Unknown Source)

                at org.apache.derby.client.am.Connection.closeResources(Unknown Source)

                - locked <0xfffffffc91831e18> (a org.apache.derby.client.net.NetConnection40)

                at org.apache.derby.client.net.NetConnection.closeResources(Unknown Source)

                - locked <0xfffffffc91831e18> (a org.apache.derby.client.net.NetConnection40)

                at org.apache.derby.client.ClientPooledConnection.close(Unknown Source)

                - locked <0xfffffffc91831cf8> (a org.apache.derby.client.ClientXAConnection40)

                at org.apache.derby.client.ClientXAConnection.close(Unknown Source)

                - locked <0xfffffffc91831cf8> (a org.apache.derby.client.ClientXAConnection40)

                at com.sun.gjc.spi.ManagedConnection.destroy(ManagedConnection.java:276)


First thing I would suggest is move up to 10.8.2.2 just to get running on the latest.
I wonder is there indication in the derby.log what went wrong here?  If not, I would suggest enabling client and server side DRDA tracing to get a clue as to what went wrong with the protocol here.
http://wiki.apache.org/db-derby/ProtocolDebuggingTips


Maybe there is some issue with the logic that dermines if autocommit will generate flow.

   // precondition: autoCommit_ is true
    void readAutoCommit() throws SqlException {
        if (willAutoCommitGenerateFlow()) {
            readCommit();
        }
    }

  public boolean willAutoCommitGenerateFlow() throws org.apache.derby.client.am.SqlException {
        if (!autoCommit_) {
            return false;
        }
        if (! allowLocalCommitRollback_()) {
            return false;
        }
        return true;
    }