activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mats Henrikson (JIRA)" <j...@apache.org>
Subject [jira] Created: (AMQ-2356) optimizeDispatch=true and UseDedicatedTaskRunner=false lead to deadlocked queues
Date Mon, 17 Aug 2009 22:53:35 GMT
optimizeDispatch=true and UseDedicatedTaskRunner=false lead to deadlocked queues
--------------------------------------------------------------------------------

                 Key: AMQ-2356
                 URL: https://issues.apache.org/activemq/browse/AMQ-2356
             Project: ActiveMQ
          Issue Type: Bug
          Components: Broker
    Affects Versions: 5.2.0
            Reporter: Mats Henrikson


We have an environment where we have a very large number of destinations. In an effort to
reduce the number of threads I have set the options

-Dorg.apache.activemq.UseDedicatedTaskRunner=false

and 

<policyEntry queue=">" optimizedDispatch="true"/>

Unfortunately this very quickly leads to deadlocked queues.

My environment is:

ActiveMQ 5.2
Ubunty Jaunty kernel 2.6.28-14-generic #47-Ubuntu SMP (although only a single core on my system)
TCP transportConnector

To reproduce the bug (which I can do 100% of the time) I connect 5 consumers (AUTO_ACK) to
5 different queues. Then I start 5 producers and pair them up with a consumer on a queue,
and they start sending PERSISTENT messages. I've set the producer to send 100 messages and
disconnect, and the consumer to receive 100 messages and disconnect. The first pair usually
gets through their 100 messages and disconnect, at which point all the other pairs have deadlocked
at less than 30 messages each.

At this point I can connect jconsole and hit the Detect Deadlock button and it finds the 4
deadlocks for me. The 4 deadlocks all have the same stacktraces, one of which looks like this:

{panel}
Name: ActiveMQ Transport: tcp:///127.0.0.1:49307
State: BLOCKED on java.lang.Object@f8828 owned by: ActiveMQ Transport: tcp:///127.0.0.1:48031
Total blocked: 3  Total waited: 1

Stack trace: 
org.apache.activemq.broker.region.PrefetchSubscription.dispatchPending(PrefetchSubscription.java:576)
org.apache.activemq.broker.region.PrefetchSubscription.add(PrefetchSubscription.java:150)
org.apache.activemq.broker.region.Queue.doActualDispatch(Queue.java:1266)
org.apache.activemq.broker.region.Queue.doDispatch(Queue.java:1230)
org.apache.activemq.broker.region.Queue.pageInMessages(Queue.java:1308)
org.apache.activemq.broker.region.Queue.iterate(Queue.java:1011)
org.apache.activemq.broker.region.Queue.wakeup(Queue.java:1146)
org.apache.activemq.broker.region.Queue.sendMessage(Queue.java:1141)
org.apache.activemq.broker.region.Queue.doMessageSend(Queue.java:474)
org.apache.activemq.broker.region.Queue.send(Queue.java:417)
org.apache.activemq.broker.region.AbstractRegion.send(AbstractRegion.java:350)
org.apache.activemq.broker.region.RegionBroker.send(RegionBroker.java:437)
org.apache.activemq.broker.TransactionBroker.send(TransactionBroker.java:224)
org.apache.activemq.broker.BrokerFilter.send(BrokerFilter.java:126)
org.apache.activemq.broker.CompositeDestinationBroker.send(CompositeDestinationBroker.java:95)
org.apache.activemq.broker.BrokerFilter.send(BrokerFilter.java:126)
...SpecialAuthorizationBroker.send(DynamicAuthorizationBroker.java:156)
org.apache.activemq.broker.BrokerFilter.send(BrokerFilter.java:126)
org.apache.activemq.broker.MutableBrokerFilter.send(MutableBrokerFilter.java:133)
org.apache.activemq.broker.TransportConnection.processMessage(TransportConnection.java:450)
org.apache.activemq.command.ActiveMQMessage.visit(ActiveMQMessage.java:639)
org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:305)
org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:179)
org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:68)
org.apache.activemq.transport.WireFormatNegotiator.onCommand(WireFormatNegotiator.java:143)
org.apache.activemq.transport.InactivityMonitor.onCommand(InactivityMonitor.java:206)
org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:84)
org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:203)
org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:185)
java.lang.Thread.run(Thread.java:595)

----------------------------------------------------------------------------------

Name: ActiveMQ Transport: tcp:///127.0.0.1:48031
State: BLOCKED on org.apache.activemq.broker.region.Queue$2@102d82c owned by: ActiveMQ Transport:
tcp:///127.0.0.1:49307
Total blocked: 3  Total waited: 0

Stack trace: 
org.apache.activemq.broker.region.Queue.iterate(Queue.java:951)
org.apache.activemq.broker.region.Queue.wakeup(Queue.java:1146)
org.apache.activemq.broker.region.Queue.removeMessage(Queue.java:1073)
org.apache.activemq.broker.region.QueueSubscription.acknowledge(QueueSubscription.java:51)
org.apache.activemq.broker.region.PrefetchSubscription.acknowledge(PrefetchSubscription.java:238)
org.apache.activemq.broker.region.AbstractRegion.acknowledge(AbstractRegion.java:373)
org.apache.activemq.broker.region.RegionBroker.acknowledge(RegionBroker.java:462)
org.apache.activemq.broker.TransactionBroker.acknowledge(TransactionBroker.java:194)
org.apache.activemq.broker.BrokerFilter.acknowledge(BrokerFilter.java:74)
org.apache.activemq.broker.BrokerFilter.acknowledge(BrokerFilter.java:74)
org.apache.activemq.broker.BrokerFilter.acknowledge(BrokerFilter.java:74)
org.apache.activemq.broker.BrokerFilter.acknowledge(BrokerFilter.java:74)
org.apache.activemq.broker.MutableBrokerFilter.acknowledge(MutableBrokerFilter.java:85)
org.apache.activemq.broker.TransportConnection.processMessageAck(TransportConnection.java:456)
org.apache.activemq.command.MessageAck.visit(MessageAck.java:205)
org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:305)
org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:179)
org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:68)
org.apache.activemq.transport.WireFormatNegotiator.onCommand(WireFormatNegotiator.java:143)
org.apache.activemq.transport.InactivityMonitor.onCommand(InactivityMonitor.java:206)
org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:84)
org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:203)
org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:185)
java.lang.Thread.run(Thread.java:595)
{panel}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message