activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gary Tully (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (AMQ-2652) ActiveMQ non-conformance to JMS Spec causing deadlock when using 3rd-party Resource Adapter
Date Tue, 09 Aug 2011 10:51:27 GMT

     [ https://issues.apache.org/jira/browse/AMQ-2652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Gary Tully resolved AMQ-2652.
-----------------------------

       Resolution: Fixed
    Fix Version/s: 5.5.0
         Assignee: Gary Tully

fixed by https://issues.apache.org/jira/browse/AMQ-2651

> ActiveMQ non-conformance to JMS Spec causing deadlock when using 3rd-party Resource Adapter
> -------------------------------------------------------------------------------------------
>
>                 Key: AMQ-2652
>                 URL: https://issues.apache.org/jira/browse/AMQ-2652
>             Project: ActiveMQ
>          Issue Type: Bug
>    Affects Versions: 5.3.0
>            Reporter: Eugene Rodos
>            Assignee: Gary Tully
>             Fix For: 5.6.0, 5.5.0
>
>         Attachments: BlockingServerSessionPoolDeadlockTest.java
>
>
> Linked to forum discussion: http://old.nabble.com/ActiveMQ-non-conformance-to-JMS-Spec-causing-deadlock-when-using-3rd-party-Resource-Adapter-tt27869447.html
> The following is an excerpt from the JMS Spec 1.1:
> {quote}
> *8.2.3 ServerSessionPool*
> 	...<snip>...
> 	If the ServerSessionPool is out of ServerSessions, the getServerSession() method
> 	may block. If a ConnectionConsumer is blocked, it cannot deliver new messages
> 	until a ServerSession is eventually returned.
> {quote}
> \\
> I am using a 3rd-party JMS Resource Adapter to integrate ActiveMQ with a J2EE app server.
 This Resource Adapter does just what the JMS Spec above allows - it blocks in the ServerSessionPool.getServerSession()
call when it is out of ServerSessions.  This causes ActiveMQ to hang due to a deadlock.
> The deadlock occurs because the ServerSessionPool uses a single connection to service
all its ServerSessions.  This is perfectly legal.  However, ActiveMQ is using this same connection
for both dispatching messages to the ConnectionConsumer as well as communicating transaction
commits to the JMS engine.  
> So the deadlock scenario is as follows:
> \- message 1 is dispatched to the ConnectionConsumer and is being processed
> \- before message 1 is committed, message 2 is dispatched
> \- ServerSession is requested for processing message 2
> \- ServerSessionPool.getServerSession() blocks because it is out of ServerSessions, blocking
the connection from which the call was made
> \- message 1 is finished processing and the tx is committed
> \- ActiveMQ does a synchronous send using the same connection that is currently blocked,
to communicate the transaction commit command
> \- the commit blocks waiting for the connection to be released which will never happen
because that release is dependent on the ServerSession being returned to the pool which won't
happen until the tx is committed
> The stacktraces of the 2 deadlocked threads at the time of the deadlock are as follows:
> {noformat} 
> "ActiveMQ Transport: tcp://localhost/127.0.0.1:61616" prio=6 tid=0x0aecc9f8 nid=0x1414
in Object.wait() [0x0b6af000..0x0b6afc68]
> 	at java.lang.Object.wait(Native Method)
> 	- waiting on <0x03070188> (a org.apache.activemq.BlockingServerSessionPoolDeadlockTest$TestServerSessionPool)
> 	at java.lang.Object.wait(Object.java:474)
> 	at org.apache.activemq.BlockingServerSessionPoolDeadlockTest$TestServerSessionPool.getServerSession(BlockingServerSessionPoolDeadlockTest.java:106)
> 	- locked <0x03070188> (a org.apache.activemq.BlockingServerSessionPoolDeadlockTest$TestServerSessionPool)
> 	at org.apache.activemq.ActiveMQConnectionConsumer.dispatch(ActiveMQConnectionConsumer.java:129)
> 	at org.apache.activemq.ActiveMQConnection$2.processMessageDispatch(ActiveMQConnection.java:1676)
> 	at org.apache.activemq.command.MessageDispatch.visit(MessageDispatch.java:108)
> 	at org.apache.activemq.ActiveMQConnection.onCommand(ActiveMQConnection.java:1658)
> 	at org.apache.activemq.transport.ResponseCorrelator.onCommand(ResponseCorrelator.java:109)
> 	at org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:68)
> 	at org.apache.activemq.transport.WireFormatNegotiator.onCommand(WireFormatNegotiator.java:113)
> 	at org.apache.activemq.transport.InactivityMonitor.onCommand(InactivityMonitor.java:210)
> 	- locked <0x03064e98> (a org.apache.activemq.transport.InactivityMonitor$1)
> 	at org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:84)
> 	at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:203)
> 	at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:185)
> 	at java.lang.Thread.run(Thread.java:595)
> 	
> "Thread-5" prio=6 tid=0x0af48ae0 nid=0x14a4 waiting on condition [0x0b8af000..0x0b8afbe8]
> 	at sun.misc.Unsafe.park(Native Method)
> 	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:118)
> 	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1841)
> 	at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:341)
> 	at org.apache.activemq.transport.FutureResponse.getResult(FutureResponse.java:40)
> 	at org.apache.activemq.transport.ResponseCorrelator.request(ResponseCorrelator.java:80)
> 	at org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1244)
> 	at org.apache.activemq.TransactionContext.syncSendPacketWithInterruptionHandling(TransactionContext.java:643)
> 	at org.apache.activemq.TransactionContext.commit(TransactionContext.java:286)
> 	at org.apache.activemq.ActiveMQSession.commit(ActiveMQSession.java:510)
> 	at org.apache.activemq.BlockingServerSessionPoolDeadlockTest$TestServerSession$1.run(BlockingServerSessionPoolDeadlockTest.java:140)
> {noformat} 
> \\
> I'm attaching a junit test that illustrates this deadlock.
> Now, I realize that this bug does not manifest itself if the ActiveMQ Resource Adapter
is used.  However, that is only because of the way AMQ RA works - it reuses existing ServerSessions
instead of blocking when it runs out (and does a weird callback from the jms session to the
ServerSession to start a tx, in order to avoid delivering several messages on same tx which
is what would happen when reusing an existing ServerSession that is already in the middle
of processing another message).  
> I think this is a bug in AMQ JMS engine's implementation which just happens to be masked
by the way AMQ RA is implemented.  If a Resource Adapted relies on the blocking behavior clearly
provisioned by the JMS Spec, the problem becomes evident. 

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message