activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eugene Rodos (JIRA)" <j...@apache.org>
Subject [jira] Created: (AMQ-2652) ActiveMQ non-conformance to JMS Spec causing deadlock when using 3rd-party Resource Adapter
Date Mon, 15 Mar 2010 21:45:49 GMT
ActiveMQ non-conformance to JMS Spec causing deadlock when using 3rd-party Resource Adapter
-------------------------------------------------------------------------------------------

                 Key: AMQ-2652
                 URL: https://issues.apache.org/activemq/browse/AMQ-2652
             Project: ActiveMQ
          Issue Type: Bug
    Affects Versions: 5.3.0
            Reporter: Eugene Rodos


Linked to forum discussion: http://old.nabble.com/ActiveMQ-non-conformance-to-JMS-Spec-causing-deadlock-when-using-3rd-party-Resource-Adapter-tt27869447.html

The following is an excerpt from the JMS Spec 1.1:

{quote}
*8.2.3 ServerSessionPool*

	...<snip>...

	If the ServerSessionPool is out of ServerSessions, the getServerSession() method
	may block. If a ConnectionConsumer is blocked, it cannot deliver new messages
	until a ServerSession is eventually returned.
{quote}
\\
I am using a 3rd-party JMS Resource Adapter to integrate ActiveMQ with a J2EE app server.
 This Resource Adapter does just what the JMS Spec above allows - it blocks in the ServerSessionPool.getServerSession()
call when it is out of ServerSessions.  This causes ActiveMQ to hang due to a deadlock.

The deadlock occurs because the ServerSessionPool uses a single connection to service all
its ServerSessions.  This is perfectly legal.  However, ActiveMQ is using this same connection
for both dispatching messages to the ConnectionConsumer as well as communicating transaction
commits to the JMS engine.  

So the deadlock scenario is as follows:
\- message 1 is dispatched to the ConnectionConsumer and is being processed
\- before message 1 is committed, message 2 is dispatched
\- ServerSession is requested for processing message 2
\- ServerSessionPool.getServerSession() blocks because it is out of ServerSessions, blocking
the connection from which the call was made
\- message 1 is finished processing and the tx is committed
\- ActiveMQ does a synchronous send using the same connection that is currently blocked, to
communicate the transaction commit command
\- the commit blocks waiting for the connection to be released which will never happen because
that release is dependent on the ServerSession being returned to the pool which won't happen
until the tx is committed

The stacktraces of the 2 deadlocked threads at the time of the deadlock are as follows:

{noformat} 
"ActiveMQ Transport: tcp://localhost/127.0.0.1:61616" prio=6 tid=0x0aecc9f8 nid=0x1414 in
Object.wait() [0x0b6af000..0x0b6afc68]
	at java.lang.Object.wait(Native Method)
	- waiting on <0x03070188> (a org.apache.activemq.BlockingServerSessionPoolDeadlockTest$TestServerSessionPool)
	at java.lang.Object.wait(Object.java:474)
	at org.apache.activemq.BlockingServerSessionPoolDeadlockTest$TestServerSessionPool.getServerSession(BlockingServerSessionPoolDeadlockTest.java:106)
	- locked <0x03070188> (a org.apache.activemq.BlockingServerSessionPoolDeadlockTest$TestServerSessionPool)
	at org.apache.activemq.ActiveMQConnectionConsumer.dispatch(ActiveMQConnectionConsumer.java:129)
	at org.apache.activemq.ActiveMQConnection$2.processMessageDispatch(ActiveMQConnection.java:1676)
	at org.apache.activemq.command.MessageDispatch.visit(MessageDispatch.java:108)
	at org.apache.activemq.ActiveMQConnection.onCommand(ActiveMQConnection.java:1658)
	at org.apache.activemq.transport.ResponseCorrelator.onCommand(ResponseCorrelator.java:109)
	at org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:68)
	at org.apache.activemq.transport.WireFormatNegotiator.onCommand(WireFormatNegotiator.java:113)
	at org.apache.activemq.transport.InactivityMonitor.onCommand(InactivityMonitor.java:210)
	- locked <0x03064e98> (a org.apache.activemq.transport.InactivityMonitor$1)
	at org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:84)
	at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:203)
	at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:185)
	at java.lang.Thread.run(Thread.java:595)
	

"Thread-5" prio=6 tid=0x0af48ae0 nid=0x14a4 waiting on condition [0x0b8af000..0x0b8afbe8]
	at sun.misc.Unsafe.park(Native Method)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:118)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1841)
	at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:341)
	at org.apache.activemq.transport.FutureResponse.getResult(FutureResponse.java:40)
	at org.apache.activemq.transport.ResponseCorrelator.request(ResponseCorrelator.java:80)
	at org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1244)
	at org.apache.activemq.TransactionContext.syncSendPacketWithInterruptionHandling(TransactionContext.java:643)
	at org.apache.activemq.TransactionContext.commit(TransactionContext.java:286)
	at org.apache.activemq.ActiveMQSession.commit(ActiveMQSession.java:510)
	at org.apache.activemq.BlockingServerSessionPoolDeadlockTest$TestServerSession$1.run(BlockingServerSessionPoolDeadlockTest.java:140)
{noformat} 

\\
I'm attaching a junit test that illustrates this deadlock.

Now, I realize that this bug does not manifest itself if the ActiveMQ Resource Adapter is
used.  However, that is only because of the way AMQ RA works - it reuses existing ServerSessions
instead of blocking when it runs out (and does a weird callback from the jms session to the
ServerSession to start a tx, in order to avoid delivering several messages on same tx which
is what would happen when reusing an existing ServerSession that is already in the middle
of processing another message).  

I think this is a bug in AMQ JMS engine's implementation which just happens to be masked by
the way AMQ RA is implemented.  If a Resource Adapted relies on the blocking behavior clearly
provisioned by the JMS Spec, the problem becomes evident. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message