I have two databases (one Derby 10.8.2.2, one Oracle 11) joining a transaction.  I'm using Atomikos as the transaction manager.  When Atomikos calls recover() with TMNOFLAGS to get the Xid(s) on the Derby branch it throws an exception.  The stacktrace is below... 

-------stacktrace---------
java.util.ConcurrentModificationException
    at java.util.WeakHashMap$HashIterator.remove(Unknown Source)
    at org.apache.derby.client.am.Connection.markStatementsClosed(Unknown Source)
    at org.apache.derby.client.am.Connection.completeChainBreakingDisconnect(Unknown Source)
    at org.apache.derby.client.am.Agent.disconnectEvent(Unknown Source)
    at org.apache.derby.client.am.DisconnectException.<init>(Unknown Source)
    at org.apache.derby.client.am.DisconnectException.<init>(Unknown Source)
    at org.apache.derby.client.am.DisconnectException.<init>(Unknown Source)
    at org.apache.derby.client.am.DisconnectException.<init>(Unknown Source)
    at org.apache.derby.client.net.Reply.doSyntaxrmSemantics(Unknown Source)
    at org.apache.derby.client.net.Reply.readDssHeader(Unknown Source)
    at org.apache.derby.client.net.Reply.startSameIdChainParse(Unknown Source)
    at org.apache.derby.client.net.NetStatementReply.readOpenQuery(Unknown Source)
    at org.apache.derby.client.net.StatementReply.readOpenQuery(Unknown Source)
    at org.apache.derby.client.net.NetStatement.readOpenQuery_(Unknown Source)
    at org.apache.derby.client.am.Statement.readOpenQuery(Unknown Source)
    at org.apache.derby.client.am.PreparedStatement.flowExecute(Unknown Source)
    at org.apache.derby.client.am.PreparedStatement.executeQueryX(Unknown Source)
    at org.apache.derby.client.am.PreparedStatement.executeQuery(Unknown Source)
    at org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:96)
    at org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:96)
    at org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:96)
...

org.apache.derby.client.am.XaException: XAER_RMFAIL : A network protocol error was encountered and the connection has been terminated: A PROTOCOL Data Stream Syntax Error was detected.  Reason: 0x3. Plaintext connection attempt to an SSL enabled server?
    at org.apache.derby.client.net.NetXAResource.throwXAException(Unknown Source)
    at org.apache.derby.client.net.NetXAResource.recover(Unknown Source)
    at com.atomikos.datasource.xa.XATransactionalResource.recover(XATransactionalResource.java:585)
    at com.atomikos.datasource.xa.XATransactionalResource.endRecovery(XATransactionalResource.java:670)
    at com.atomikos.icatch.imp.TransactionServiceImp.recover(TransactionServiceImp.java:635)
    at com.atomikos.datasource.xa.XATransactionalResource.setRecoveryService(XATransactionalResource.java:503)
    at com.atomikos.icatch.system.Configuration.addResource(Configuration.java:404)
    at com.atomikos.icatch.jta.TransactionImp.enlistResource(TransactionImp.java:334)
    at org.apache.commons.dbcp.managed.TransactionContext.setSharedConnection(TransactionContext.java:89)
    at org.apache.commons.dbcp.managed.ManagedConnection.updateTransactionStatus(ManagedConnection.java:131)
    at org.apache.commons.dbcp.managed.ManagedConnection.<init>(ManagedConnection.java:55)
    at org.apache.commons.dbcp.managed.ManagedDataSource.getConnection(ManagedDataSource.java:77)
    at org.apache.commons.dbcp.BasicDataSource.getConnection(BasicDataSource.java:1044)
...

Even though it says there is a protocol error, I think that's a red herring because the server isn't SSL enabled.  I'm not sure if the real cause is because the Derby branch at the time is only doing a select query and so the status is XA_RDONLY but the conditional at the end of the recover() method is only checking for XA_OK and throwing the exception for all other cases or if it is because the transaction branches are in separate threads and Derby is "going first" and already running the query when the TM enlists the second database and calls recover() on all branches to enumerate the XID(s).  The same scenario seems to work fine with MySQL, PostgreSQL, etc. Is the Derby client thread safe in a XA transaction?

Thanks,
-Jason