Return-Path: X-Original-To: apmail-activemq-dev-archive@www.apache.org Delivered-To: apmail-activemq-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id AA8A517E20 for ; Fri, 26 Sep 2014 10:53:39 +0000 (UTC) Received: (qmail 10667 invoked by uid 500); 26 Sep 2014 10:53:34 -0000 Delivered-To: apmail-activemq-dev-archive@activemq.apache.org Received: (qmail 10597 invoked by uid 500); 26 Sep 2014 10:53:34 -0000 Mailing-List: contact dev-help@activemq.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@activemq.apache.org Delivered-To: mailing list dev@activemq.apache.org Received: (qmail 10585 invoked by uid 99); 26 Sep 2014 10:53:34 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 26 Sep 2014 10:53:34 +0000 Date: Fri, 26 Sep 2014 10:53:34 +0000 (UTC) From: "Gary Tully (JIRA)" To: dev@activemq.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (AMQ-5260) Cross talk over duplex network connection can lead to blocking (alternative take) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/AMQ-5260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14149020#comment-14149020 ] Gary Tully commented on AMQ-5260: --------------------------------- @matteo - i have not had the opportunity to delve into this... but for the mutex transport - my understanding is that is there to protect the socket writes - there can be multiple sessions/producers on a connection and they all want to send data concurrently over the socket. Openwire allows interleaving at the command level only. In addition there openwire cache depends on serial inserts on both ends of the channel, the mutex protects that also. inversion of lock order is a bug. It is great that you are investigating this in such detail, thanks. This code has evolved quite a bit so the more eyes on it the better. > Cross talk over duplex network connection can lead to blocking (alternative take) > --------------------------------------------------------------------------------- > > Key: AMQ-5260 > URL: https://issues.apache.org/jira/browse/AMQ-5260 > Project: ActiveMQ > Issue Type: Bug > Affects Versions: 5.9.0, 5.10.0 > Reporter: matteo rulli > Attachments: AMQ5260AdvancedTest.java, AMQ_5260.patch, AMQ_5260_2.patch, AMQ_5260_3.patch, deadlock.jpg, debug.jpg > > > Pretty the same description with respect to AMQ-4328. > ---- > !deadlock.jpg! > h2. Stacktraces: > Stacktrace no.1: > {noformat} > Name: ActiveMQ NIO Worker 12 > State: BLOCKED on java.net.URI@1bae2b28 owned by: ActiveMQ Transport: tcp:///10.0.1.219:61616@57789 > Total blocked: 2 Total waited: 67 > Stack trace: > org.apache.activemq.network.DemandForwardingBridgeSupport.serviceRemoteConsumerAdvisory(DemandForwardingBridgeSupport.java:714) > org.apache.activemq.network.DemandForwardingBridgeSupport.serviceRemoteCommand(DemandForwardingBridgeSupport.java:581) > org.apache.activemq.network.DemandForwardingBridgeSupport$3.onCommand(DemandForwardingBridgeSupport.java:191) > org.apache.activemq.transport.ResponseCorrelator.onCommand(ResponseCorrelator.java:116) > org.apache.activemq.transport.MutexTransport.onCommand(MutexTransport.java:50) > org.apache.activemq.transport.WireFormatNegotiator.onCommand(WireFormatNegotiator.java:113) > org.apache.activemq.transport.AbstractInactivityMonitor.onCommand(AbstractInactivityMonitor.java:270) > org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:83) > org.apache.activemq.transport.nio.NIOTransport.serviceRead(NIOTransport.java:138) > org.apache.activemq.transport.nio.NIOTransport$1.onSelect(NIOTransport.java:69) > org.apache.activemq.transport.nio.SelectorSelection.onSelect(SelectorSelection.java:94) > org.apache.activemq.transport.nio.SelectorWorker$1.run(SelectorWorker.java:119) > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) > java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) > java.lang.Thread.run(Unknown Source) > {noformat} > ---- > stack trace no.2 > {noformat} > Name: ActiveMQ Transport: tcp:///10.0.1.219:61616@57789 > State: WAITING on java.util.concurrent.locks.ReentrantLock$NonfairSync@3cdbfa3e owned by: ActiveMQ BrokerService[master2] Task-4 > Total blocked: 19 Total waited: 3 > Stack trace: > sun.misc.Unsafe.park(Native Method) > java.util.concurrent.locks.LockSupport.park(Unknown Source) > java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(Unknown Source) > java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(Unknown Source) > java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(Unknown Source) > java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(Unknown Source) > java.util.concurrent.locks.ReentrantLock.lock(Unknown Source) > org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:66) > org.apache.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelator.java:60) > org.apache.activemq.broker.TransportConnection.dispatch(TransportConnection.java:1339) > org.apache.activemq.broker.TransportConnection.processDispatch(TransportConnection.java:858) > org.apache.activemq.broker.TransportConnection.dispatchSync(TransportConnection.java:818) > org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:151) > org.apache.activemq.transport.ResponseCorrelator.onCommand(ResponseCorrelator.java:116) > org.apache.activemq.transport.MutexTransport.onCommand(MutexTransport.java:50) > org.apache.activemq.transport.vm.VMTransport.doDispatch(VMTransport.java:138) > org.apache.activemq.transport.vm.VMTransport.dispatch(VMTransport.java:127) > - locked java.util.concurrent.atomic.AtomicBoolean@689389da > org.apache.activemq.transport.vm.VMTransport.oneway(VMTransport.java:104) > org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:68) > org.apache.activemq.transport.ResponseCorrelator.asyncRequest(ResponseCorrelator.java:81) > org.apache.activemq.transport.ResponseCorrelator.request(ResponseCorrelator.java:86) > org.apache.activemq.network.DemandForwardingBridgeSupport.addSubscription(DemandForwardingBridgeSupport.java:856) > org.apache.activemq.network.DemandForwardingBridgeSupport.addConsumerInfo(DemandForwardingBridgeSupport.java:1128) > org.apache.activemq.network.DemandForwardingBridgeSupport.serviceRemoteConsumerAdvisory(DemandForwardingBridgeSupport.java:714) > - locked java.net.URI@1bae2b28 > org.apache.activemq.network.DemandForwardingBridgeSupport.serviceRemoteCommand(DemandForwardingBridgeSupport.java:581) > org.apache.activemq.network.DemandForwardingBridgeSupport$3.onCommand(DemandForwardingBridgeSupport.java:191) > org.apache.activemq.transport.ResponseCorrelator.onCommand(ResponseCorrelator.java:116) > org.apache.activemq.transport.MutexTransport.onCommand(MutexTransport.java:50) > org.apache.activemq.transport.failover.FailoverTransport$3.onCommand(FailoverTransport.java:196) > org.apache.activemq.transport.WireFormatNegotiator.onCommand(WireFormatNegotiator.java:113) > org.apache.activemq.transport.AbstractInactivityMonitor.onCommand(AbstractInactivityMonitor.java:270) > org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:83) > org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:214) > org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:196) > java.lang.Thread.run(Unknown Source) > {noformat} > ---- > stack trace no.3 > {noformat} > Name: ActiveMQ BrokerService[master2] Task-4 > State: WAITING on java.util.concurrent.locks.ReentrantLock$NonfairSync@717bb9c owned by: ActiveMQ Transport: tcp:///10.0.1.219:61616@57789 > Total blocked: 19 Total waited: 117 > Stack trace: > sun.misc.Unsafe.park(Native Method) > java.util.concurrent.locks.LockSupport.park(Unknown Source) > java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(Unknown Source) > java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(Unknown Source) > java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(Unknown Source) > java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(Unknown Source) > java.util.concurrent.locks.ReentrantLock.lock(Unknown Source) > org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:66) > org.apache.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelator.java:60) > org.apache.activemq.network.DemandForwardingBridgeSupport.serviceLocalCommand(DemandForwardingBridgeSupport.java:921) > org.apache.activemq.network.DemandForwardingBridgeSupport$2.onCommand(DemandForwardingBridgeSupport.java:173) > org.apache.activemq.transport.ResponseCorrelator.onCommand(ResponseCorrelator.java:116) > org.apache.activemq.transport.MutexTransport.onCommand(MutexTransport.java:50) > org.apache.activemq.transport.vm.VMTransport.doDispatch(VMTransport.java:138) > org.apache.activemq.transport.vm.VMTransport.dispatch(VMTransport.java:127) > - locked java.util.concurrent.atomic.AtomicBoolean@453bb109 > org.apache.activemq.transport.vm.VMTransport.oneway(VMTransport.java:104) > org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:68) > org.apache.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelator.java:60) > org.apache.activemq.broker.TransportConnection.dispatch(TransportConnection.java:1339) > org.apache.activemq.broker.TransportConnection.processDispatch(TransportConnection.java:858) > org.apache.activemq.broker.TransportConnection.iterate(TransportConnection.java:904) > org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:129) > org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:47) > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) > java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) > java.lang.Thread.run(Unknown Source) > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)