activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Timothy Bish (JIRA)" <j...@apache.org>
Subject [jira] [Closed] (AMQ-2749) Broker hangs in TopicSubscription.add
Date Mon, 11 Jul 2011 21:23:59 GMT

     [ https://issues.apache.org/jira/browse/AMQ-2749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Timothy Bish closed AMQ-2749.
-----------------------------

       Resolution: Not A Problem
    Fix Version/s:     (was: 5.6.0)

Configuration issue, broker was reaching its memory limits.  

> Broker hangs in TopicSubscription.add
> -------------------------------------
>
>                 Key: AMQ-2749
>                 URL: https://issues.apache.org/jira/browse/AMQ-2749
>             Project: ActiveMQ
>          Issue Type: Bug
>          Components: Broker
>    Affects Versions: 5.3.1
>         Environment: red had enterprise
>            Reporter: Dennis Klinkott
>         Attachments: activemq.xml, jmx console screenshot.jpg, jstack.2010-05-25-14-01-30,
jstack.2010-05-25-14-10-54
>
>
> Our broker suddenly hang in production environment. We created a bunch of jstacks to
see what the reason is.
> The jstacks show that the broker waits with many threads in doMessageSend:
> e.g.
> "ActiveMQ Transport: tcp:///10.231.233.117:51088" daemon prio=10 tid=0x00002aab84016800
nid=0x1f19 waiting for monitor entry [0x000000006a8f2000]
>    java.lang.Thread.State: BLOCKED (on object monitor)
> 	at org.apache.activemq.broker.region.Topic.doMessageSend(Topic.java:402)
> 	- waiting to lock <0x00002aaab46f8938> (a org.apache.activemq.broker.region.Topic)
> 	at org.apache.activemq.broker.region.Topic.send(Topic.java:384)
> 	at org.apache.activemq.broker.region.DestinationFilter.send(DestinationFilter.java:93)
> 	at org.apache.activemq.broker.region.virtual.VirtualTopicInterceptor.send(VirtualTopicInterceptor.java:46)
> 	at org.apache.activemq.broker.region.AbstractRegion.send(AbstractRegion.java:354)
> 	at org.apache.activemq.broker.region.RegionBroker.send(RegionBroker.java:445)
> 	at org.apache.activemq.broker.TransactionBroker.send(TransactionBroker.java:224)
> 	at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:443)
> 	at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:373)
> 	at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:368)
> 	at org.apache.activemq.advisory.AdvisoryBroker.addDestinationInfo(AdvisoryBroker.java:173)
> 	at org.apache.activemq.broker.BrokerFilter.addDestinationInfo(BrokerFilter.java:214)
> 	at org.apache.activemq.broker.BrokerFilter.addDestinationInfo(BrokerFilter.java:214)
> 	at org.apache.activemq.broker.MutableBrokerFilter.addDestinationInfo(MutableBrokerFilter.java:221)
> 	at org.apache.activemq.broker.TransportConnection.processAddDestination(TransportConnection.java:467)
> 	at org.apache.activemq.command.DestinationInfo.visit(DestinationInfo.java:122)
> 	at org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:300)
> 	at org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:178)
> 	at org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:68)
> 	at org.apache.activemq.transport.stomp.StompTransportFilter.sendToActiveMQ(StompTransportFilter.java:82)
> 	at org.apache.activemq.transport.stomp.ProtocolConverter.sendToActiveMQ(ProtocolConverter.java:135)
> 	at org.apache.activemq.transport.stomp.ProtocolConverter.createTempQueue(ProtocolConverter.java:611)
> 	at org.apache.activemq.transport.stomp.LegacyFrameTranslator.convertDestination(LegacyFrameTranslator.java:130)
> 	at org.apache.activemq.transport.stomp.FrameTranslator$Helper.copyStandardHeadersFromFrameToMessage(FrameTranslator.java:93)
> 	at org.apache.activemq.transport.stomp.LegacyFrameTranslator.convertFrame(LegacyFrameTranslator.java:54)
> 	at org.apache.activemq.transport.stomp.ProtocolConverter.convertMessage(ProtocolConverter.java:591)
> 	at org.apache.activemq.transport.stomp.ProtocolConverter.onStompSend(ProtocolConverter.java:231)
> 	at org.apache.activemq.transport.stomp.ProtocolConverter.onStompCommand(ProtocolConverter.java:173)
> 	at org.apache.activemq.transport.stomp.StompTransportFilter.onCommand(StompTransportFilter.java:71)
> 	at org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:84)
> 	at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:204)
> 	at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:186)
> 	at java.lang.Thread.run(Thread.java:619)
> It is waiting to lock 0x00002aaab46f8938. This is the thread that holds the lock:
> "ActiveMQ Transport: tcp:///10.231.233.117:43020" daemon prio=10 tid=0x00000000608d8800
nid=0xc6e in Object.wait() [0x000000007d41c000]
>    java.lang.Thread.State: TIMED_WAITING (on object monitor)
> 	at java.lang.Object.wait(Native Method)
> 	at org.apache.activemq.broker.region.TopicSubscription.add(TopicSubscription.java:106)
> 	- locked <0x00002aaab57d34e8> (a java.lang.Object)
> 	at org.apache.activemq.broker.region.policy.SimpleDispatchPolicy.dispatch(SimpleDispatchPolicy.java:49)
> 	- locked <0x00002aaab46f98e0> (a java.util.concurrent.CopyOnWriteArrayList)
> 	at org.apache.activemq.broker.region.Topic.dispatch(Topic.java:585)
> 	at org.apache.activemq.broker.region.Topic.doMessageSend(Topic.java:443)
> 	- locked <0x00002aaab46f8938> (a org.apache.activemq.broker.region.Topic)
> 	at org.apache.activemq.broker.region.Topic.send(Topic.java:384)
> 	at org.apache.activemq.broker.region.DestinationFilter.send(DestinationFilter.java:93)
> 	at org.apache.activemq.broker.region.virtual.VirtualTopicInterceptor.send(VirtualTopicInterceptor.java:46)
> 	at org.apache.activemq.broker.region.AbstractRegion.send(AbstractRegion.java:354)
> 	at org.apache.activemq.broker.region.RegionBroker.send(RegionBroker.java:445)
> 	at org.apache.activemq.broker.TransactionBroker.send(TransactionBroker.java:224)
> 	at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:443)
> 	at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:373)
> 	at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:368)
> 	at org.apache.activemq.advisory.AdvisoryBroker.removeDestination(AdvisoryBroker.java:185)
> 	at org.apache.activemq.broker.BrokerFilter.removeDestination(BrokerFilter.java:146)
> 	at org.apache.activemq.broker.BrokerFilter.removeDestination(BrokerFilter.java:146)
> 	at org.apache.activemq.broker.MutableBrokerFilter.removeDestination(MutableBrokerFilter.java:153)
> 	at org.apache.activemq.broker.TransportConnection.processRemoveConnection(TransportConnection.java:713)
> 	- locked <0x00002aaac32b10c0> (a org.apache.activemq.broker.jmx.ManagedTransportConnection)
> 	at org.apache.activemq.command.RemoveInfo.visit(RemoveInfo.java:72)
> 	at org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:300)
> 	at org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:178)
> 	at org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:68)
> 	at org.apache.activemq.transport.stomp.StompTransportFilter.sendToActiveMQ(StompTransportFilter.java:82)
> 	at org.apache.activemq.transport.stomp.ProtocolConverter.sendToActiveMQ(ProtocolConverter.java:135)
> 	at org.apache.activemq.transport.stomp.ProtocolConverter.onStompDisconnect(ProtocolConverter.java:545)
> 	at org.apache.activemq.transport.stomp.ProtocolConverter.onStompCommand(ProtocolConverter.java:189)
> 	at org.apache.activemq.transport.stomp.StompTransportFilter.onCommand(StompTransportFilter.java:71)
> 	at org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:84)
> 	at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:204)
> 	at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:186)
> 	at java.lang.Thread.run(Thread.java:619)
> So it is the client 10.231.233.117 that has a stomp-connection to the broker, causes
a TopicSubscription.add and causes the broker freeze. 
> This is how we configure our openwire transport-connector in the activemq.xml:
>         <transportConnectors>
>             <transportConnector name="openwire" uri="tcp://0.0.0.0:61616"/>
>             <transportConnector name="stomp" uri="stomp://0.0.0.0:61613"/>
>         </transportConnectors>
> We took jstack-dumps with a distance of 9 minutes. Both are attached. In both dumps it
is the same TopicSubscription.add that causes the hang.
> How can we solve this issue? Looks like the connection should run into an timeout but
doesn't. Is this an activemq bug? Can we somehow set the broker openwire timeout parameters?
> Thanks for any hints.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message