Return-Path: X-Original-To: apmail-activemq-issues-archive@minotaur.apache.org Delivered-To: apmail-activemq-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 209CD184FF for ; Thu, 3 Sep 2015 22:09:46 +0000 (UTC) Received: (qmail 24762 invoked by uid 500); 3 Sep 2015 22:09:46 -0000 Delivered-To: apmail-activemq-issues-archive@activemq.apache.org Received: (qmail 24735 invoked by uid 500); 3 Sep 2015 22:09:46 -0000 Mailing-List: contact issues-help@activemq.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@activemq.apache.org Delivered-To: mailing list issues@activemq.apache.org Received: (qmail 24724 invoked by uid 99); 3 Sep 2015 22:09:45 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 03 Sep 2015 22:09:45 +0000 Date: Thu, 3 Sep 2015 22:09:45 +0000 (UTC) From: "ASF subversion and git services (JIRA)" To: issues@activemq.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (ARTEMIS-217) Deadlock between Topology updates and connection creation MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/ARTEMIS-217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14729889#comment-14729889 ] ASF subversion and git services commented on ARTEMIS-217: --------------------------------------------------------- Commit 220e39ef1fa0173f2da9772c5e4911681bbd056f in activemq-artemis's branch refs/heads/master from Clebert Suconic [ https://git-wip-us.apache.org/repos/asf?p=activemq-artemis.git;h=220e39e ] ARTEMIS-217 fixing dead lock https://issues.apache.org/jira/browse/ARTEMIS-217 fixing dead lock This is using a separate lock for notifications, this way we won't hold a lock while communicating on netty which was the issue here. > Deadlock between Topology updates and connection creation > --------------------------------------------------------- > > Key: ARTEMIS-217 > URL: https://issues.apache.org/jira/browse/ARTEMIS-217 > Project: ActiveMQ Artemis > Issue Type: Bug > Affects Versions: 1.0.0 > Reporter: clebert suconic > Assignee: clebert suconic > Fix For: 1.1.0 > > > This was a rare case, it only happened in one specific environment: > "Thread-5 (ActiveMQ-client-netty-threads-1937173143)" #315 daemon prio=5 os_prio=0 tid=0x00007f5b5c020000 nid=0x361d waiting for monitor entry [0x00007f5bb9587000] > java.lang.Thread.State: BLOCKED (on object monitor) > at org.apache.activemq.artemis.core.client.impl.ServerLocatorImpl.notifyNodeUp(ServerLocatorImpl.java:1413) > - waiting to lock <0x000000008832bbf8> (a org.apache.activemq.artemis.core.client.impl.ServerLocatorImpl) > at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$SessionFactoryTopologyHandler.notifyNodeUp(ClientSessionFactoryImpl.java:1275) > at org.apache.activemq.artemis.core.protocol.core.impl.ActiveMQClientProtocolManager$Channel0Handler.notifyTopologyChange(ActiveMQClientProtocolManager.java:491) > at org.apache.activemq.artemis.core.protocol.core.impl.ActiveMQClientProtocolManager$Channel0Handler.handlePacket(ActiveMQClientProtocolManager.java:445) > at org.apache.activemq.artemis.core.protocol.core.impl.ChannelImpl.handlePacket(ChannelImpl.java:559) > at org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.doBufferReceived(RemotingConnectionImpl.java:349) > - locked <0x0000000087145738> (a java.lang.Object) > at org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.bufferReceived(RemotingConnectionImpl.java:331) > at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$DelegatingBufferHandler.bufferReceived(ClientSessionFactoryImpl.java:1097) > at org.apache.activemq.artemis.core.remoting.impl.netty.ActiveMQChannelHandler.channelRead(ActiveMQChannelHandler.java:68) > at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308) > at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:294) > at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:244) > at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308) > at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:294) > at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:846) > at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131) > at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511) > at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468) > at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382) > at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354) > at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111) > at java.lang.Thread.run(Thread.java:745) > versus: > "Thread-1 (ActiveMQ-server-ActiveMQServerImpl::Server 4-1766145591)" #466 prio=5 os_prio=0 tid=0x00007f5bf4f54000 nid=0x367d in Object.wait() [0x00007f5b5249e000] > java.lang.Thread.State: WAITING (on object monitor) > at java.lang.Object.wait(Native Method) > - waiting on <0x00000000e81ca680> (a io.netty.channel.DefaultChannelPromise) > at java.lang.Object.wait(Object.java:502) > at io.netty.util.concurrent.DefaultPromise.awaitUninterruptibly(DefaultPromise.java:286) > - locked <0x00000000e81ca680> (a io.netty.channel.DefaultChannelPromise) > at io.netty.channel.DefaultChannelPromise.awaitUninterruptibly(DefaultChannelPromise.java:135) > at io.netty.channel.DefaultChannelPromise.awaitUninterruptibly(DefaultChannelPromise.java:28) > at org.apache.activemq.artemis.core.remoting.impl.netty.NettyConnector.createConnection(NettyConnector.java:604) > at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.openTransportConnection(ClientSessionFactoryImpl.java:982) > at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.createTransportConnection(ClientSessionFactoryImpl.java:1026) > at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.establishNewConnection(ClientSessionFactoryImpl.java:1205) > at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.getConnection(ClientSessionFactoryImpl.java:843) > - locked <0x00000000e685c1b0> (a java.lang.Object) > at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.getConnectionWithRetry(ClientSessionFactoryImpl.java:745) > at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.connect(ClientSessionFactoryImpl.java:231) > at org.apache.activemq.artemis.core.client.impl.ServerLocatorImpl.createSessionFactory(ServerLocatorImpl.java:791) > - locked <0x000000008832bbf8> (a org.apache.activemq.artemis.core.client.impl.ServerLocatorImpl) > at org.apache.activemq.artemis.core.client.impl.ServerLocatorImpl.connect(ServerLocatorImpl.java:649) > at org.apache.activemq.artemis.core.client.impl.ServerLocatorImpl.connect(ServerLocatorImpl.java:636) > at org.apache.activemq.artemis.core.client.impl.ServerLocatorImpl$4.run(ServerLocatorImpl.java:1479) > at org.apache.activemq.artemis.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:94) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) -- This message was sent by Atlassian JIRA (v6.3.4#6332)