activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From davidk <davidklosow...@livenation.com>
Subject ActiveMQ Transport Threads Getting Blocked
Date Tue, 27 Aug 2013 02:50:45 GMT
I'm having issues with one of our activemq servers.  Many of the transport
threads are blocking after the server comes up and clients connect to it.

The ActiveMQ version is 5.4.2 with quite a few back ported patches.

There are many blocked transport threads and the snippet below just shows
the blocked thread and the running thread that is blocking it.  Judging by
the snippet from below, it looks like the Transport thread on
ssl:///10.73.216.103:51640 has the transport mutex and is not releasing it. 
It looks to be writing to a socket as a result of a dispatch but somehow
stuck.  

Does anyone have suggestions as to what may be causing this?

"ActiveMQ Transport: ssl:///10.73.111.127:49523" - Thread t@7246
   java.lang.Thread.State: BLOCKED
	at
org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:40)
	- waiting to lock <7fed4b35> (a java.lang.Object) owned by "ActiveMQ
Transport: ssl:///10.73.216.103:51640" t@663
	at
org.apache.activemq.broker.TransportConnection.dispatch(TransportConnection.java:1284)
	at
org.apache.activemq.broker.TransportConnection.processDispatch(TransportConnection.java:822)
	at
org.apache.activemq.broker.TransportConnection.dispatchSync(TransportConnection.java:782)
	at
org.apache.activemq.broker.region.TopicSubscription.dispatch(TopicSubscription.java:525)
	at
org.apache.activemq.broker.region.TopicSubscription.add(TopicSubscription.java:99)
	at
org.apache.activemq.broker.region.policy.SimpleDispatchPolicy.dispatch(SimpleDispatchPolicy.java:48)
	at org.apache.activemq.broker.region.Topic.dispatch(Topic.java:657)
	at org.apache.activemq.broker.region.Topic.doMessageSend(Topic.java:489)
	- locked <3bfdff07> (a org.apache.activemq.broker.region.Topic)
	at org.apache.activemq.broker.region.Topic.send(Topic.java:425)
	at
org.apache.activemq.broker.region.AbstractRegion.send(AbstractRegion.java:365)
	at
org.apache.activemq.broker.region.RegionBroker.send(RegionBroker.java:518)
	at
org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:491)
	at
org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:422)
	at
org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:417)
	at
org.apache.activemq.advisory.AdvisoryBroker.addDestinationInfo(AdvisoryBroker.java:186)
	at
org.apache.activemq.broker.BrokerFilter.addDestinationInfo(BrokerFilter.java:221)
	at
org.apache.activemq.broker.BrokerFilter.addDestinationInfo(BrokerFilter.java:221)
	at
org.apache.activemq.broker.BrokerFilter.addDestinationInfo(BrokerFilter.java:221)
	at
org.apache.activemq.broker.BrokerFilter.addDestinationInfo(BrokerFilter.java:221)
	at
org.apache.activemq.broker.BrokerFilter.addDestinationInfo(BrokerFilter.java:221)
	at
com.ticketmaster.activemq.security.IdentityBrokerFilter.addDestinationInfo(IdentityBrokerFilter.java:473)
	at
org.apache.activemq.broker.BrokerFilter.addDestinationInfo(BrokerFilter.java:221)
	at
org.apache.activemq.broker.BrokerFilter.addDestinationInfo(BrokerFilter.java:221)
	at
org.apache.activemq.broker.BrokerFilter.addDestinationInfo(BrokerFilter.java:221)
	at
org.apache.activemq.broker.BrokerFilter.addDestinationInfo(BrokerFilter.java:221)
	at
org.apache.activemq.broker.MutableBrokerFilter.addDestinationInfo(MutableBrokerFilter.java:227)
	at
org.apache.activemq.broker.TransportConnection.processAddDestination(TransportConnection.java:485)
	at
org.apache.activemq.command.DestinationInfo.visit(DestinationInfo.java:122)
	at
org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:312)
	at
org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:186)
	at
org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:69)
	at
org.apache.activemq.transport.WireFormatNegotiator.onCommand(WireFormatNegotiator.java:113)
	at
org.apache.activemq.transport.InactivityMonitor.onCommand(InactivityMonitor.java:228)
	- locked <5f718b0b> (a org.apache.activemq.transport.InactivityMonitor$1)
	at
org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:83)
	at
org.apache.activemq.transport.tcp.SslTransport.doConsume(SslTransport.java:91)
	at
org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:220)
	at
org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:202)
	at java.lang.Thread.run(Thread.java:662)

   Locked ownable synchronizers:
	- None


"ActiveMQ Transport: ssl:///10.73.216.103:51640" - Thread t@663
   java.lang.Thread.State: RUNNABLE
	at java.net.SocketOutputStream.socketWrite0(Native Method)
	at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
	at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
	at
com.sun.net.ssl.internal.ssl.OutputRecord.writeBuffer(OutputRecord.java:358)
	at com.sun.net.ssl.internal.ssl.OutputRecord.write(OutputRecord.java:346)
	at
com.sun.net.ssl.internal.ssl.SSLSocketImpl.writeRecordInternal(SSLSocketImpl.java:781)
	at
com.sun.net.ssl.internal.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:753)
	at
com.sun.net.ssl.internal.ssl.AppOutputStream.write(AppOutputStream.java:100)
	- locked <48a49bda> (a com.sun.net.ssl.internal.ssl.AppOutputStream)
	at
org.apache.activemq.transport.tcp.TcpBufferedOutputStream.flush(TcpBufferedOutputStream.java:115)
	at java.io.DataOutputStream.flush(DataOutputStream.java:106)
	at
org.apache.activemq.transport.tcp.TcpTransport.oneway(TcpTransport.java:182)
	at
org.apache.activemq.transport.InactivityMonitor.oneway(InactivityMonitor.java:255)
	- locked <2ef42a78> (a java.util.concurrent.atomic.AtomicBoolean)
	at
org.apache.activemq.transport.TransportFilter.oneway(TransportFilter.java:85)
	at
org.apache.activemq.transport.WireFormatNegotiator.oneway(WireFormatNegotiator.java:104)
	at
org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:40)
	- locked <7fed4b35> (a java.lang.Object)
	at
org.apache.activemq.broker.TransportConnection.dispatch(TransportConnection.java:1284)
	at
org.apache.activemq.broker.TransportConnection.processDispatch(TransportConnection.java:822)
	at
org.apache.activemq.broker.TransportConnection.dispatchSync(TransportConnection.java:782)
	at
org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:188)
	at
org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:69)
	at
org.apache.activemq.transport.WireFormatNegotiator.onCommand(WireFormatNegotiator.java:113)
	at
org.apache.activemq.transport.InactivityMonitor.onCommand(InactivityMonitor.java:228)
	- locked <2010445d> (a org.apache.activemq.transport.InactivityMonitor$1)
	at
org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:83)
	at
org.apache.activemq.transport.tcp.SslTransport.doConsume(SslTransport.java:91)
	at
org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:220)
	at
org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:202)
	at java.lang.Thread.run(Thread.java:662)

   Locked ownable synchronizers:
	- locked <bf8461d> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)





--
View this message in context: http://activemq.2283324.n4.nabble.com/ActiveMQ-Transport-Threads-Getting-Blocked-tp4670703.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Mime
View raw message