activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christian Posta <christian.po...@gmail.com>
Subject Re: ActiveMQ Transport Threads Getting Blocked
Date Tue, 27 Aug 2013 03:02:34 GMT
Yep, looks like you might have some slow topic consumers.


On Mon, Aug 26, 2013 at 7:50 PM, davidk <davidklosowski@livenation.com>wrote:

> I'm having issues with one of our activemq servers.  Many of the transport
> threads are blocking after the server comes up and clients connect to it.
>
> The ActiveMQ version is 5.4.2 with quite a few back ported patches.
>
> There are many blocked transport threads and the snippet below just shows
> the blocked thread and the running thread that is blocking it.  Judging by
> the snippet from below, it looks like the Transport thread on
> ssl:///10.73.216.103:51640 has the transport mutex and is not releasing
> it.
> It looks to be writing to a socket as a result of a dispatch but somehow
> stuck.
>
> Does anyone have suggestions as to what may be causing this?
>
> "ActiveMQ Transport: ssl:///10.73.111.127:49523" - Thread t@7246
>    java.lang.Thread.State: BLOCKED
>         at
> org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:40)
>         - waiting to lock <7fed4b35> (a java.lang.Object) owned by
> "ActiveMQ
> Transport: ssl:///10.73.216.103:51640" t@663
>         at
>
> org.apache.activemq.broker.TransportConnection.dispatch(TransportConnection.java:1284)
>         at
>
> org.apache.activemq.broker.TransportConnection.processDispatch(TransportConnection.java:822)
>         at
>
> org.apache.activemq.broker.TransportConnection.dispatchSync(TransportConnection.java:782)
>         at
>
> org.apache.activemq.broker.region.TopicSubscription.dispatch(TopicSubscription.java:525)
>         at
>
> org.apache.activemq.broker.region.TopicSubscription.add(TopicSubscription.java:99)
>         at
>
> org.apache.activemq.broker.region.policy.SimpleDispatchPolicy.dispatch(SimpleDispatchPolicy.java:48)
>         at org.apache.activemq.broker.region.Topic.dispatch(Topic.java:657)
>         at
> org.apache.activemq.broker.region.Topic.doMessageSend(Topic.java:489)
>         - locked <3bfdff07> (a org.apache.activemq.broker.region.Topic)
>         at org.apache.activemq.broker.region.Topic.send(Topic.java:425)
>         at
>
> org.apache.activemq.broker.region.AbstractRegion.send(AbstractRegion.java:365)
>         at
> org.apache.activemq.broker.region.RegionBroker.send(RegionBroker.java:518)
>         at
>
> org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:491)
>         at
>
> org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:422)
>         at
>
> org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:417)
>         at
>
> org.apache.activemq.advisory.AdvisoryBroker.addDestinationInfo(AdvisoryBroker.java:186)
>         at
>
> org.apache.activemq.broker.BrokerFilter.addDestinationInfo(BrokerFilter.java:221)
>         at
>
> org.apache.activemq.broker.BrokerFilter.addDestinationInfo(BrokerFilter.java:221)
>         at
>
> org.apache.activemq.broker.BrokerFilter.addDestinationInfo(BrokerFilter.java:221)
>         at
>
> org.apache.activemq.broker.BrokerFilter.addDestinationInfo(BrokerFilter.java:221)
>         at
>
> org.apache.activemq.broker.BrokerFilter.addDestinationInfo(BrokerFilter.java:221)
>         at
>
> com.ticketmaster.activemq.security.IdentityBrokerFilter.addDestinationInfo(IdentityBrokerFilter.java:473)
>         at
>
> org.apache.activemq.broker.BrokerFilter.addDestinationInfo(BrokerFilter.java:221)
>         at
>
> org.apache.activemq.broker.BrokerFilter.addDestinationInfo(BrokerFilter.java:221)
>         at
>
> org.apache.activemq.broker.BrokerFilter.addDestinationInfo(BrokerFilter.java:221)
>         at
>
> org.apache.activemq.broker.BrokerFilter.addDestinationInfo(BrokerFilter.java:221)
>         at
>
> org.apache.activemq.broker.MutableBrokerFilter.addDestinationInfo(MutableBrokerFilter.java:227)
>         at
>
> org.apache.activemq.broker.TransportConnection.processAddDestination(TransportConnection.java:485)
>         at
> org.apache.activemq.command.DestinationInfo.visit(DestinationInfo.java:122)
>         at
>
> org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:312)
>         at
>
> org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:186)
>         at
>
> org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:69)
>         at
>
> org.apache.activemq.transport.WireFormatNegotiator.onCommand(WireFormatNegotiator.java:113)
>         at
>
> org.apache.activemq.transport.InactivityMonitor.onCommand(InactivityMonitor.java:228)
>         - locked <5f718b0b> (a
> org.apache.activemq.transport.InactivityMonitor$1)
>         at
>
> org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:83)
>         at
>
> org.apache.activemq.transport.tcp.SslTransport.doConsume(SslTransport.java:91)
>         at
> org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:220)
>         at
> org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:202)
>         at java.lang.Thread.run(Thread.java:662)
>
>    Locked ownable synchronizers:
>         - None
>
>
> "ActiveMQ Transport: ssl:///10.73.216.103:51640" - Thread t@663
>    java.lang.Thread.State: RUNNABLE
>         at java.net.SocketOutputStream.socketWrite0(Native Method)
>         at
> java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
>         at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
>         at
>
> com.sun.net.ssl.internal.ssl.OutputRecord.writeBuffer(OutputRecord.java:358)
>         at
> com.sun.net.ssl.internal.ssl.OutputRecord.write(OutputRecord.java:346)
>         at
>
> com.sun.net.ssl.internal.ssl.SSLSocketImpl.writeRecordInternal(SSLSocketImpl.java:781)
>         at
>
> com.sun.net.ssl.internal.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:753)
>         at
>
> com.sun.net.ssl.internal.ssl.AppOutputStream.write(AppOutputStream.java:100)
>         - locked <48a49bda> (a
> com.sun.net.ssl.internal.ssl.AppOutputStream)
>         at
>
> org.apache.activemq.transport.tcp.TcpBufferedOutputStream.flush(TcpBufferedOutputStream.java:115)
>         at java.io.DataOutputStream.flush(DataOutputStream.java:106)
>         at
>
> org.apache.activemq.transport.tcp.TcpTransport.oneway(TcpTransport.java:182)
>         at
>
> org.apache.activemq.transport.InactivityMonitor.oneway(InactivityMonitor.java:255)
>         - locked <2ef42a78> (a java.util.concurrent.atomic.AtomicBoolean)
>         at
>
> org.apache.activemq.transport.TransportFilter.oneway(TransportFilter.java:85)
>         at
>
> org.apache.activemq.transport.WireFormatNegotiator.oneway(WireFormatNegotiator.java:104)
>         at
> org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:40)
>         - locked <7fed4b35> (a java.lang.Object)
>         at
>
> org.apache.activemq.broker.TransportConnection.dispatch(TransportConnection.java:1284)
>         at
>
> org.apache.activemq.broker.TransportConnection.processDispatch(TransportConnection.java:822)
>         at
>
> org.apache.activemq.broker.TransportConnection.dispatchSync(TransportConnection.java:782)
>         at
>
> org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:188)
>         at
>
> org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:69)
>         at
>
> org.apache.activemq.transport.WireFormatNegotiator.onCommand(WireFormatNegotiator.java:113)
>         at
>
> org.apache.activemq.transport.InactivityMonitor.onCommand(InactivityMonitor.java:228)
>         - locked <2010445d> (a
> org.apache.activemq.transport.InactivityMonitor$1)
>         at
>
> org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:83)
>         at
>
> org.apache.activemq.transport.tcp.SslTransport.doConsume(SslTransport.java:91)
>         at
> org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:220)
>         at
> org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:202)
>         at java.lang.Thread.run(Thread.java:662)
>
>    Locked ownable synchronizers:
>         - locked <bf8461d> (a
> java.util.concurrent.locks.ReentrantLock$NonfairSync)
>
>
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/ActiveMQ-Transport-Threads-Getting-Blocked-tp4670703.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>



-- 
*Christian Posta*
http://www.christianposta.com/blog
twitter: @christianposta

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message