Return-Path: Delivered-To: apmail-activemq-dev-archive@www.apache.org Received: (qmail 66896 invoked from network); 23 Jul 2007 10:38:12 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 23 Jul 2007 10:38:12 -0000 Received: (qmail 64880 invoked by uid 500); 23 Jul 2007 10:38:13 -0000 Delivered-To: apmail-activemq-dev-archive@activemq.apache.org Received: (qmail 64717 invoked by uid 500); 23 Jul 2007 10:38:12 -0000 Mailing-List: contact dev-help@activemq.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@activemq.apache.org Delivered-To: mailing list dev@activemq.apache.org Received: (qmail 64707 invoked by uid 99); 23 Jul 2007 10:38:12 -0000 Received: from herse.apache.org (HELO herse.apache.org) (140.211.11.133) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 23 Jul 2007 03:38:12 -0700 X-ASF-Spam-Status: No, hits=-100.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [140.211.11.4] (HELO brutus.apache.org) (140.211.11.4) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 23 Jul 2007 03:38:09 -0700 Received: from brutus (localhost [127.0.0.1]) by brutus.apache.org (Postfix) with ESMTP id 2430E71404B for ; Mon, 23 Jul 2007 03:37:49 -0700 (PDT) Message-ID: <32351451.1185187069143.JavaMail.jira@brutus> Date: Mon, 23 Jul 2007 03:37:49 -0700 (PDT) From: "Rob Davies (JIRA)" To: dev@activemq.apache.org Subject: [jira] Assigned: (AMQ-897) Deadlock in ActiveMQ Failover Worker MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/activemq/browse/AMQ-897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rob Davies reassigned AMQ-897: ------------------------------ Assignee: Rob Davies > Deadlock in ActiveMQ Failover Worker > ------------------------------------ > > Key: AMQ-897 > URL: https://issues.apache.org/activemq/browse/AMQ-897 > Project: ActiveMQ > Issue Type: Bug > Components: Broker > Affects Versions: 4.0.1 > Environment: Linux 2.6.9-34.ELsmp #1 SMP Fri Feb 24 16:54:53 EST 2006 i686 i686 i386 GNU/Linux > Dell PE 850 with 4 GB RAM and 1x dual core 3GHz Pentium D > Java HotSpot(TM) Client VM (build 1.5.0_06-b05, mixed mode, sharing) > Reporter: Michael P Lum > Assignee: Rob Davies > Fix For: 5.0.0 > > > Full thread dump Java HotSpot(TM) Client VM (1.5.0_06-b05 mixed mode, sharing): > "ActiveMQ Transport: tcp://XXX/XXX:61616" prio=1 tid=0x08091510 nid=0x2f4c waiting for monitor entry [0xafdc9000..0xafdca130] > at org.apache.activemq.network.DemandForwardingBridgeSupport$2.transportInterupted(DemandForwardingBridgeSupport.java:134) > - waiting to lock <0x3654b290> (a org.apache.activemq.network.DemandForwardingBridgeSupport$2) > at org.apache.activemq.transport.TransportFilter.transportInterupted(TransportFilter.java:98) > at org.apache.activemq.transport.TransportFilter.transportInterupted(TransportFilter.java:98) > at org.apache.activemq.transport.failover.FailoverTransport.handleTransportFailure(FailoverTransport.java:223) > at org.apache.activemq.transport.failover.FailoverTransport.access$300(FailoverTransport.java:53) > at org.apache.activemq.transport.failover.FailoverTransport$1.onException(FailoverTransport.java:111) > at org.apache.activemq.transport.TransportFilter.onException(TransportFilter.java:94) > at org.apache.activemq.transport.WireFormatNegotiator.onException(WireFormatNegotiator.java:120) > at org.apache.activemq.transport.InactivityMonitor.onException(InactivityMonitor.java:149) > at org.apache.activemq.transport.TransportSupport.onException(TransportSupport.java:100) > at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:156) > at java.lang.Thread.run(Thread.java:595) > "Thread-7" daemon prio=1 tid=0x085263d8 nid=0x2f4b waiting for monitor entry [0xafe4b000..0xafe4b1b0] > at org.apache.activemq.transport.failover.FailoverTransport.oneway(FailoverTransport.java:335) > - waiting to lock <0x365793b0> (a java.lang.Object) > at org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:44) > - locked <0x36579610> (a java.lang.Object) > at org.apache.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelator.java:60) > at org.apache.activemq.network.DemandForwardingBridgeSupport.startRemoteBridge(DemandForwardingBridgeSupport.java:235) > at org.apache.activemq.network.DemandForwardingBridgeSupport$4.run(DemandForwardingBridgeSupport.java:197) > "ActiveMQ Transport: tcp:///127.0.0.1:40496" daemon prio=1 tid=0x0851c070 nid=0x2f3c runnable [0xafac3000..0xafac3fb0] > at java.net.SocketInputStream.socketRead0(Native Method) > at java.net.SocketInputStream.read(SocketInputStream.java:129) > at org.apache.activemq.transport.tcp.TcpBufferedInputStream.fill(TcpBufferedInputStream.java:48) > at org.apache.activemq.transport.tcp.TcpBufferedInputStream.read(TcpBufferedInputStream.java:55) > at java.io.DataInputStream.readInt(DataInputStream.java:353) > at org.apache.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:274) > at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:142) > at java.lang.Thread.run(Thread.java:595) > "ActiveMQ Connection Dispatcher: 9847682" daemon prio=1 tid=0x0851bc30 nid=0x2f3b in Object.wait() [0xafb44000..0xafb45030] > at java.lang.Object.wait(Native Method) > at java.lang.Object.wait(Object.java:474) > at org.apache.activemq.thread.DedicatedTaskRunner.runTask(DedicatedTaskRunner.java:91) > - locked <0x37849a98> (a java.lang.Object) > at org.apache.activemq.thread.DedicatedTaskRunner.access$000(DedicatedTaskRunner.java:24) > at org.apache.activemq.thread.DedicatedTaskRunner$1.run(DedicatedTaskRunner.java:38) > "ActiveMQ Transport: tcp:///127.0.0.1:40386" daemon prio=1 tid=0x0855b060 nid=0x2b0e runnable [0xb06db000..0xb06dc1b0] > at java.net.SocketInputStream.socketRead0(Native Method) > at java.net.SocketInputStream.read(SocketInputStream.java:129) > at org.apache.activemq.transport.tcp.TcpBufferedInputStream.fill(TcpBufferedInputStream.java:48) > at org.apache.activemq.transport.tcp.TcpBufferedInputStream.read(TcpBufferedInputStream.java:55) > at java.io.DataInputStream.readInt(DataInputStream.java:353) > at org.apache.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:274) > at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:142) > at java.lang.Thread.run(Thread.java:595) > "ActiveMQ Connection Dispatcher: 11854491" daemon prio=1 tid=0x085110d8 nid=0x2b0d in Object.wait() [0xafbc5000..0xafbc5e30] > at java.lang.Object.wait(Native Method) > - waiting on <0x36406180> (a java.lang.Object) > at java.lang.Object.wait(Object.java:474) > at org.apache.activemq.thread.DedicatedTaskRunner.runTask(DedicatedTaskRunner.java:91) > - locked <0x36406180> (a java.lang.Object) > at org.apache.activemq.thread.DedicatedTaskRunner.access$000(DedicatedTaskRunner.java:24) > at org.apache.activemq.thread.DedicatedTaskRunner$1.run(DedicatedTaskRunner.java:38) > "ActiveMQ Transport: tcp:///127.0.0.1:40385" daemon prio=1 tid=0x0855bfe0 nid=0x2b09 runnable [0xb0af6000..0xb0af6eb0] > at java.net.SocketInputStream.socketRead0(Native Method) > at java.net.SocketInputStream.read(SocketInputStream.java:129) > at org.apache.activemq.transport.tcp.TcpBufferedInputStream.fill(TcpBufferedInputStream.java:48) > at org.apache.activemq.transport.tcp.TcpBufferedInputStream.read(TcpBufferedInputStream.java:55) > at java.io.DataInputStream.readInt(DataInputStream.java:353) > at org.apache.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:274) > at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:142) > at java.lang.Thread.run(Thread.java:595) > "ActiveMQ Connection Dispatcher: 14732323" daemon prio=1 tid=0x085822e8 nid=0x2b07 in Object.wait() [0xb0bf8000..0xb0bf8f30] > at java.lang.Object.wait(Native Method) > - waiting on <0x36408358> (a java.lang.Object) > at java.lang.Object.wait(Object.java:474) > at org.apache.activemq.thread.DedicatedTaskRunner.runTask(DedicatedTaskRunner.java:91) > - locked <0x36408358> (a java.lang.Object) > at org.apache.activemq.thread.DedicatedTaskRunner.access$000(DedicatedTaskRunner.java:24) > at org.apache.activemq.thread.DedicatedTaskRunner$1.run(DedicatedTaskRunner.java:38) > "ActiveMQ Transport: tcp:///127.0.0.1:40381" daemon prio=1 tid=0x0855aec0 nid=0x2af1 runnable [0xb0a75000..0xb0a75fb0] > at java.net.SocketInputStream.socketRead0(Native Method) > at java.net.SocketInputStream.read(SocketInputStream.java:129) > at org.apache.activemq.transport.tcp.TcpBufferedInputStream.fill(TcpBufferedInputStream.java:48) > at org.apache.activemq.transport.tcp.TcpBufferedInputStream.read(TcpBufferedInputStream.java:55) > at java.io.DataInputStream.readInt(DataInputStream.java:353) > at org.apache.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:274) > at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:142) > at java.lang.Thread.run(Thread.java:595) > "ActiveMQ Connection Dispatcher: 2804837" daemon prio=1 tid=0x0850ad00 nid=0x2af0 in Object.wait() [0xb0cfc000..0xb0cfd030] > at java.lang.Object.wait(Native Method) > - waiting on <0x3640a530> (a java.lang.Object) > at java.lang.Object.wait(Object.java:474) > at org.apache.activemq.thread.DedicatedTaskRunner.runTask(DedicatedTaskRunner.java:91) > - locked <0x3640a530> (a java.lang.Object) > at org.apache.activemq.thread.DedicatedTaskRunner.access$000(DedicatedTaskRunner.java:24) > at org.apache.activemq.thread.DedicatedTaskRunner$1.run(DedicatedTaskRunner.java:38) > "ActiveMQ Transport: tcp:///10.7.93.117:47626" daemon prio=1 tid=0x08092be8 nid=0x2ad9 runnable [0xafcc7000..0xafcc81b0] > at java.net.SocketInputStream.socketRead0(Native Method) > at java.net.SocketInputStream.read(SocketInputStream.java:129) > at org.apache.activemq.transport.tcp.TcpBufferedInputStream.fill(TcpBufferedInputStream.java:48) > at org.apache.activemq.transport.tcp.TcpBufferedInputStream.read(TcpBufferedInputStream.java:55) > at java.io.DataInputStream.readInt(DataInputStream.java:353) > at org.apache.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:274) > at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:142) > at java.lang.Thread.run(Thread.java:595) > "ActiveMQ Connection Dispatcher: 5165021" daemon prio=1 tid=0x08092600 nid=0x2ad8 in Object.wait() [0xafd48000..0xafd48e30] > at java.lang.Object.wait(Native Method) > - waiting on <0x3640c708> (a java.lang.Object) > at java.lang.Object.wait(Object.java:474) > at org.apache.activemq.thread.DedicatedTaskRunner.runTask(DedicatedTaskRunner.java:91) > - locked <0x3640c708> (a java.lang.Object) > at org.apache.activemq.thread.DedicatedTaskRunner.access$000(DedicatedTaskRunner.java:24) > at org.apache.activemq.thread.DedicatedTaskRunner$1.run(DedicatedTaskRunner.java:38) > "ActiveMQ Transport: tcp:///XXX:49524" daemon prio=1 tid=0x08094ef0 nid=0x2a9d runnable [0xafecb000..0xafecbfb0] > at java.net.SocketInputStream.socketRead0(Native Method) > at java.net.SocketInputStream.read(SocketInputStream.java:129) > at org.apache.activemq.transport.tcp.TcpBufferedInputStream.fill(TcpBufferedInputStream.java:48) > at org.apache.activemq.transport.tcp.TcpBufferedInputStream.read(TcpBufferedInputStream.java:55) > at java.io.DataInputStream.readInt(DataInputStream.java:353) > at org.apache.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:274) > at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:142) > at java.lang.Thread.run(Thread.java:595) > "ActiveMQ Connection Dispatcher: 10884088" daemon prio=1 tid=0x08093c20 nid=0x2a9c in Object.wait() [0xaff4c000..0xaff4d030] > at java.lang.Object.wait(Native Method) > - waiting on <0x3640e8e0> (a java.lang.Object) > at java.lang.Object.wait(Object.java:474) > at org.apache.activemq.thread.DedicatedTaskRunner.runTask(DedicatedTaskRunner.java:91) > - locked <0x3640e8e0> (a java.lang.Object) > at org.apache.activemq.thread.DedicatedTaskRunner.access$000(DedicatedTaskRunner.java:24) > at org.apache.activemq.thread.DedicatedTaskRunner$1.run(DedicatedTaskRunner.java:38) > "ActiveMQ Transport: tcp:///XXX:49523" daemon prio=1 tid=0x08093e90 nid=0x2a9b runnable [0xaffcd000..0xaffce0b0] > at java.net.SocketInputStream.socketRead0(Native Method) > at java.net.SocketInputStream.read(SocketInputStream.java:129) > at org.apache.activemq.transport.tcp.TcpBufferedInputStream.fill(TcpBufferedInputStream.java:48) > at org.apache.activemq.transport.tcp.TcpBufferedInputStream.read(TcpBufferedInputStream.java:55) > at java.io.DataInputStream.readInt(DataInputStream.java:353) > at org.apache.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:274) > at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:142) > at java.lang.Thread.run(Thread.java:595) > "ActiveMQ Connection Dispatcher: 30810736" daemon prio=1 tid=0x08526ca8 nid=0x2a9a in Object.wait() [0xb004e000..0xb004f130] > at java.lang.Object.wait(Native Method) > - waiting on <0x36410ab8> (a java.lang.Object) > at java.lang.Object.wait(Object.java:474) > at org.apache.activemq.thread.DedicatedTaskRunner.runTask(DedicatedTaskRunner.java:91) > - locked <0x36410ab8> (a java.lang.Object) > at org.apache.activemq.thread.DedicatedTaskRunner.access$000(DedicatedTaskRunner.java:24) > at org.apache.activemq.thread.DedicatedTaskRunner$1.run(DedicatedTaskRunner.java:38) > "ActiveMQ Scheduler" daemon prio=1 tid=0x08524730 nid=0x2a99 waiting for monitor entry [0xb00cf000..0xb00d01b0] > at org.apache.activemq.network.DemandForwardingBridgeSupport$2.transportInterupted(DemandForwardingBridgeSupport.java:134) > - waiting to lock <0x3654b290> (a org.apache.activemq.network.DemandForwardingBridgeSupport$2) > at org.apache.activemq.transport.TransportFilter.transportInterupted(TransportFilter.java:98) > at org.apache.activemq.transport.TransportFilter.transportInterupted(TransportFilter.java:98) > at org.apache.activemq.transport.failover.FailoverTransport.handleTransportFailure(FailoverTransport.java:223) > at org.apache.activemq.transport.failover.FailoverTransport.access$300(FailoverTransport.java:53) > at org.apache.activemq.transport.failover.FailoverTransport$1.onException(FailoverTransport.java:111) > at org.apache.activemq.transport.TransportFilter.onException(TransportFilter.java:94) > at org.apache.activemq.transport.WireFormatNegotiator.onException(WireFormatNegotiator.java:120) > at org.apache.activemq.transport.InactivityMonitor.onException(InactivityMonitor.java:149) > at org.apache.activemq.transport.InactivityMonitor.readCheck(InactivityMonitor.java:101) > at org.apache.activemq.transport.InactivityMonitor.access$000(InactivityMonitor.java:35) > at org.apache.activemq.transport.InactivityMonitor$1.run(InactivityMonitor.java:51) > at edu.emory.mathcs.backport.java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:431) > at edu.emory.mathcs.backport.java.util.concurrent.FutureTask.runAndReset(FutureTask.java:198) > at edu.emory.mathcs.backport.java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:189) > at edu.emory.mathcs.backport.java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:213) > at edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650) > at edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675) > at java.lang.Thread.run(Thread.java:595) > "ActiveMQ Transport: tcp:///XXX:49522" daemon prio=1 tid=0x08520f28 nid=0x2a98 runnable [0xb0150000..0xb0150e30] > at java.net.SocketInputStream.socketRead0(Native Method) > at java.net.SocketInputStream.read(SocketInputStream.java:129) > at org.apache.activemq.transport.tcp.TcpBufferedInputStream.fill(TcpBufferedInputStream.java:48) > at org.apache.activemq.transport.tcp.TcpBufferedInputStream.read(TcpBufferedInputStream.java:55) > at java.io.DataInputStream.readInt(DataInputStream.java:353) > at org.apache.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:274) > at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:142) > at java.lang.Thread.run(Thread.java:595) > "ActiveMQ Connection Dispatcher: 9363478" daemon prio=1 tid=0x0851d400 nid=0x2a97 in Object.wait() [0xb01d1000..0xb01d1eb0] > at java.lang.Object.wait(Native Method) > - waiting on <0x36412dd0> (a java.lang.Object) > at java.lang.Object.wait(Object.java:474) > at org.apache.activemq.thread.DedicatedTaskRunner.runTask(DedicatedTaskRunner.java:91) > - locked <0x36412dd0> (a java.lang.Object) > at org.apache.activemq.thread.DedicatedTaskRunner.access$000(DedicatedTaskRunner.java:24) > at org.apache.activemq.thread.DedicatedTaskRunner$1.run(DedicatedTaskRunner.java:38) > "ActiveMQ Scheduler" daemon prio=1 tid=0x08527a88 nid=0x2a96 in Object.wait() [0xb0252000..0xb0252f30] > at java.lang.Object.wait(Native Method) > at java.lang.Object.wait(Object.java:432) > at edu.emory.mathcs.backport.java.util.concurrent.TimeUnit.timedWait(TimeUnit.java:301) > at edu.emory.mathcs.backport.java.util.concurrent.DelayQueue.take(DelayQueue.java:156) > - locked <0x36410b38> (a java.lang.Object) > at edu.emory.mathcs.backport.java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:590) > at edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:477) > at edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:674) > at java.lang.Thread.run(Thread.java:595) > "ActiveMQ Scheduler" daemon prio=1 tid=0x08527360 nid=0x2a95 in Object.wait() [0xb02d3000..0xb02d3fb0] > at java.lang.Object.wait(Native Method) > at java.lang.Object.wait(Object.java:432) > at edu.emory.mathcs.backport.java.util.concurrent.TimeUnit.timedWait(TimeUnit.java:301) > at edu.emory.mathcs.backport.java.util.concurrent.DelayQueue.take(DelayQueue.java:156) > - locked <0x36410b38> (a java.lang.Object) > at edu.emory.mathcs.backport.java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:590) > at edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:477) > at edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:674) > at java.lang.Thread.run(Thread.java:595) > "ActiveMQ Transport: tcp:///XXX:49521" daemon prio=1 tid=0x0851fc98 nid=0x2a94 runnable [0xb0354000..0xb0355030] > at java.net.SocketInputStream.socketRead0(Native Method) > at java.net.SocketInputStream.read(SocketInputStream.java:129) > at org.apache.activemq.transport.tcp.TcpBufferedInputStream.fill(TcpBufferedInputStream.java:48) > at org.apache.activemq.transport.tcp.TcpBufferedInputStream.read(TcpBufferedInputStream.java:55) > at java.io.DataInputStream.readInt(DataInputStream.java:353) > at org.apache.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:274) > at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:142) > at java.lang.Thread.run(Thread.java:595) > "ActiveMQ Connection Dispatcher: 15779934" daemon prio=1 tid=0x0851f950 nid=0x2a93 in Object.wait() [0xb04d7000..0xb04d80b0] > at java.lang.Object.wait(Native Method) > - waiting on <0x364150a8> (a java.lang.Object) > at java.lang.Object.wait(Object.java:474) > at org.apache.activemq.thread.DedicatedTaskRunner.runTask(DedicatedTaskRunner.java:91) > - locked <0x364150a8> (a java.lang.Object) > at org.apache.activemq.thread.DedicatedTaskRunner.access$000(DedicatedTaskRunner.java:24) > at org.apache.activemq.thread.DedicatedTaskRunner$1.run(DedicatedTaskRunner.java:38) > "ActiveMQ Scheduler" daemon prio=1 tid=0x083acc68 nid=0x2a90 in Object.wait() [0xb03d5000..0xb03d61b0] > at java.lang.Object.wait(Native Method) > at java.lang.Object.wait(Object.java:432) > at edu.emory.mathcs.backport.java.util.concurrent.TimeUnit.timedWait(TimeUnit.java:301) > at edu.emory.mathcs.backport.java.util.concurrent.DelayQueue.take(DelayQueue.java:156) > - locked <0x36410b38> (a java.lang.Object) > at edu.emory.mathcs.backport.java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:590) > at edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:477) > at edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:674) > at java.lang.Thread.run(Thread.java:595) > "ActiveMQ Scheduler" daemon prio=1 tid=0x083ac700 nid=0x2a8f in Object.wait() [0xb0456000..0xb0456e30] > at java.lang.Object.wait(Native Method) > at java.lang.Object.wait(Object.java:432) > at edu.emory.mathcs.backport.java.util.concurrent.TimeUnit.timedWait(TimeUnit.java:301) > at edu.emory.mathcs.backport.java.util.concurrent.DelayQueue.take(DelayQueue.java:156) > - locked <0x36410b38> (a java.lang.Object) > at edu.emory.mathcs.backport.java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:590) > at edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:477) > at edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:674) > at java.lang.Thread.run(Thread.java:595) > "ActiveMQ Transport: tcp://XXX/10.7.93.117:61616" prio=1 tid=0x083ad3d0 nid=0x2a8d runnable [0xb0558000..0xb0558f30] > at java.net.SocketInputStream.socketRead0(Native Method) > at java.net.SocketInputStream.read(SocketInputStream.java:129) > at org.apache.activemq.transport.tcp.TcpBufferedInputStream.fill(TcpBufferedInputStream.java:48) > at org.apache.activemq.transport.tcp.TcpBufferedInputStream.read(TcpBufferedInputStream.java:55) > at java.io.DataInputStream.readInt(DataInputStream.java:353) > at org.apache.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:274) > at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:142) > at java.lang.Thread.run(Thread.java:595) > "ActiveMQ Failover Worker: 2891371" daemon prio=1 tid=0xb0e0f1c0 nid=0x2a8c in Object.wait() [0xb05d9000..0xb05d9fb0] > at java.lang.Object.wait(Native Method) > - waiting on <0x36417380> (a java.lang.Object) > at java.lang.Object.wait(Object.java:474) > at org.apache.activemq.thread.DedicatedTaskRunner.runTask(DedicatedTaskRunner.java:91) > - locked <0x36417380> (a java.lang.Object) > at org.apache.activemq.thread.DedicatedTaskRunner.access$000(DedicatedTaskRunner.java:24) > at org.apache.activemq.thread.DedicatedTaskRunner$1.run(DedicatedTaskRunner.java:38) > "ActiveMQ Connection Dispatcher: 21350489" daemon prio=1 tid=0xb0e0d788 nid=0x2a8b in Object.wait() [0xb065a000..0xb065b030] > at java.lang.Object.wait(Native Method) > - waiting on <0x36417400> (a java.lang.Object) > at java.lang.Object.wait(Object.java:474) > at org.apache.activemq.thread.DedicatedTaskRunner.runTask(DedicatedTaskRunner.java:91) > - locked <0x36417400> (a java.lang.Object) > at org.apache.activemq.thread.DedicatedTaskRunner.access$000(DedicatedTaskRunner.java:24) > at org.apache.activemq.thread.DedicatedTaskRunner$1.run(DedicatedTaskRunner.java:38) > "ActiveMQ Failover Worker: 5862378" daemon prio=1 tid=0xb0e0ea00 nid=0x2a89 waiting for monitor entry [0xb076f000..0xb0770130] > at org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:43) > - waiting to lock <0x36579610> (a java.lang.Object) > at org.apache.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelator.java:60) > at org.apache.activemq.network.DemandForwardingBridgeSupport$2.transportResumed(DemandForwardingBridgeSupport.java:164) > - locked <0x3654b290> (a org.apache.activemq.network.DemandForwardingBridgeSupport$2) > at org.apache.activemq.transport.TransportFilter.transportResumed(TransportFilter.java:102) > at org.apache.activemq.transport.TransportFilter.transportResumed(TransportFilter.java:102) > at org.apache.activemq.transport.failover.FailoverTransport$2.iterate(FailoverTransport.java:175) > - locked <0x365793b0> (a java.lang.Object) > at org.apache.activemq.thread.DedicatedTaskRunner.runTask(DedicatedTaskRunner.java:87) > at org.apache.activemq.thread.DedicatedTaskRunner.access$000(DedicatedTaskRunner.java:24) > at org.apache.activemq.thread.DedicatedTaskRunner$1.run(DedicatedTaskRunner.java:38) > "ActiveMQ Connection Dispatcher: 12115695" daemon prio=1 tid=0xb0e0a718 nid=0x2a88 in Object.wait() [0xb07f0000..0xb07f11b0] > at java.lang.Object.wait(Native Method) > at java.lang.Object.wait(Object.java:474) > at edu.emory.mathcs.backport.java.util.concurrent.CountDownLatch.await(CountDownLatch.java:179) > - locked <0x382fb880> (a edu.emory.mathcs.backport.java.util.concurrent.CountDownLatch) > at org.apache.activemq.network.DemandForwardingBridgeSupport.waitStarted(DemandForwardingBridgeSupport.java:836) > at org.apache.activemq.network.DemandForwardingBridgeSupport.serviceLocalCommand(DemandForwardingBridgeSupport.java:462) > at org.apache.activemq.network.DemandForwardingBridgeSupport$1.onCommand(DemandForwardingBridgeSupport.java:116) > at org.apache.activemq.transport.ResponseCorrelator.onCommand(ResponseCorrelator.java:97) > at org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:63) > at org.apache.activemq.transport.vm.VMTransport.oneway(VMTransport.java:76) > at org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:44) > - locked <0x36417648> (a java.lang.Object) > at org.apache.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelator.java:60) > at org.apache.activemq.broker.TransportConnection.dispatch(TransportConnection.java:211) > at org.apache.activemq.broker.AbstractConnection.processDispatch(AbstractConnection.java:581) > at org.apache.activemq.broker.AbstractConnection.iterate(AbstractConnection.java:597) > at org.apache.activemq.thread.DedicatedTaskRunner.runTask(DedicatedTaskRunner.java:87) > at org.apache.activemq.thread.DedicatedTaskRunner.access$000(DedicatedTaskRunner.java:24) > at org.apache.activemq.thread.DedicatedTaskRunner$1.run(DedicatedTaskRunner.java:38) > "ActiveMQ Transport Status Monitor: vm://localhost" daemon prio=1 tid=0xb0e08090 nid=0x2a87 waiting on condition [0xb0871000..0xb0871e30] > at java.lang.Thread.sleep(Native Method) > at org.apache.activemq.broker.TransportStatusDetector.run(TransportStatusDetector.java:98) > at java.lang.Thread.run(Thread.java:595) > "ActiveMQ Transport Status Monitor: stomp" daemon prio=1 tid=0xb0e01fb0 nid=0x2a86 waiting on condition [0xb08f2000..0xb08f2eb0] > at java.lang.Thread.sleep(Native Method) > at org.apache.activemq.broker.TransportStatusDetector.run(TransportStatusDetector.java:98) > at java.lang.Thread.run(Thread.java:595) > "ActiveMQ Transport Server: tcp://XXX:61613?wireFormat=stomp" daemon prio=1 tid=0xb0e01ab0 nid=0x2a85 runnable [0xb0973000..0xb0973f30] > at java.net.PlainSocketImpl.socketAccept(Native Method) > at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384) > - locked <0x36417840> (a java.net.SocksSocketImpl) > at java.net.ServerSocket.implAccept(ServerSocket.java:450) > at java.net.ServerSocket.accept(ServerSocket.java:421) > at org.apache.activemq.transport.tcp.TcpTransportServer.run(TcpTransportServer.java:123) > at java.lang.Thread.run(Thread.java:595) > "ActiveMQ Transport Status Monitor: default" daemon prio=1 tid=0xb0e00f48 nid=0x2a84 waiting on condition [0xb09f4000..0xb09f4fb0] > at java.lang.Thread.sleep(Native Method) > at org.apache.activemq.broker.TransportStatusDetector.run(TransportStatusDetector.java:98) > at java.lang.Thread.run(Thread.java:595) > "ActiveMQ Transport Server: tcp://XXX:61616" daemon prio=1 tid=0xb0e01630 nid=0x2a83 runnable [0xb1061000..0xb1062030] > at java.net.PlainSocketImpl.socketAccept(Native Method) > at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384) > - locked <0x36417a10> (a java.net.SocksSocketImpl) > at java.net.ServerSocket.implAccept(ServerSocket.java:450) > at java.net.ServerSocket.accept(ServerSocket.java:421) > at org.apache.activemq.transport.tcp.TcpTransportServer.run(TcpTransportServer.java:123) > at java.lang.Thread.run(Thread.java:595) > "RMI LeaseChecker" daemon prio=1 tid=0x08586318 nid=0x2a80 waiting on condition [0xb0b78000..0xb0b781b0] > at java.lang.Thread.sleep(Native Method) > at sun.rmi.transport.DGCImpl$LeaseChecker.run(DGCImpl.java:310) > at java.lang.Thread.run(Thread.java:595) > "RMI RenewClean-[XXX:40355]" daemon prio=1 tid=0x0850cd08 nid=0x2a7e in Object.wait() [0xb0c79000..0xb0c79eb0] > at java.lang.Object.wait(Native Method) > at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116) > - locked <0x36417bc0> (a java.lang.ref.ReferenceQueue$Lock) > at sun.rmi.transport.DGCClient$EndpointEntry$RenewCleanThread.run(DGCClient.java:500) > at java.lang.Thread.run(Thread.java:595) > "GC Daemon" daemon prio=1 tid=0x0857f9d0 nid=0x2a7c in Object.wait() [0xb0d7d000..0xb0d7dfb0] > at java.lang.Object.wait(Native Method) > - waiting on <0x364035b8> (a sun.misc.GC$LatencyLock) > at sun.misc.GC$Daemon.run(GC.java:100) > - locked <0x364035b8> (a sun.misc.GC$LatencyLock) > "RMI Reaper" prio=1 tid=0x0857ea58 nid=0x2a7b in Object.wait() [0xb0dfe000..0xb0dff030] > at java.lang.Object.wait(Native Method) > - waiting on <0x363ffae8> (a java.lang.ref.ReferenceQueue$Lock) > at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116) > - locked <0x363ffae8> (a java.lang.ref.ReferenceQueue$Lock) > at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:132) > at sun.rmi.transport.ObjectTable$Reaper.run(ObjectTable.java:336) > at java.lang.Thread.run(Thread.java:595) > "RMI TCP Accept-0" daemon prio=1 tid=0x0857b1c8 nid=0x2a7a runnable [0xb0fe0000..0xb0fe10b0] > at java.net.PlainSocketImpl.socketAccept(Native Method) > at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384) > - locked <0x36403220> (a java.net.SocksSocketImpl) > at java.net.ServerSocket.implAccept(ServerSocket.java:450) > at java.net.ServerSocket.accept(ServerSocket.java:421) > at sun.rmi.transport.tcp.TCPTransport.run(TCPTransport.java:334) > at java.lang.Thread.run(Thread.java:595) > "RMI TCP Accept-1099" daemon prio=1 tid=0x084fae08 nid=0x2a78 runnable [0xb10e2000..0xb10e31b0] > at java.net.PlainSocketImpl.socketAccept(Native Method) > at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384) > - locked <0x36400498> (a java.net.SocksSocketImpl) > at java.net.ServerSocket.implAccept(ServerSocket.java:450) > at java.net.ServerSocket.accept(ServerSocket.java:421) > at sun.rmi.transport.tcp.TCPTransport.run(TCPTransport.java:334) > at java.lang.Thread.run(Thread.java:595) > "Timer-0" daemon prio=1 tid=0x084fa780 nid=0x2a77 in Object.wait() [0xb1163000..0xb1163e30] > at java.lang.Object.wait(Native Method) > - waiting on <0x363ffc10> (a java.util.TaskQueue) > at java.lang.Object.wait(Object.java:474) > at java.util.TimerThread.mainLoop(Timer.java:483) > - locked <0x363ffc10> (a java.util.TaskQueue) > at java.util.TimerThread.run(Timer.java:462) > "Low Memory Detector" daemon prio=1 tid=0x080a3688 nid=0x2a75 runnable [0x00000000..0x00000000] > "CompilerThread0" daemon prio=1 tid=0x080a20e0 nid=0x2a74 waiting on condition [0x00000000..0xb1e52998] > "Signal Dispatcher" daemon prio=1 tid=0x080a1168 nid=0x2a73 runnable [0x00000000..0x00000000] > "Finalizer" daemon prio=1 tid=0x0809a3d8 nid=0x2a72 in Object.wait() [0xb2154000..0xb21550b0] > at java.lang.Object.wait(Native Method) > at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116) > - locked <0x36390738> (a java.lang.ref.ReferenceQueue$Lock) > at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:132) > at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159) > "Reference Handler" daemon prio=1 tid=0x08099680 nid=0x2a71 in Object.wait() [0xb21d6000..0xb21d6130] > at java.lang.Object.wait(Native Method) > at java.lang.Object.wait(Object.java:474) > at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116) > - locked <0x36390680> (a java.lang.ref.Reference$Lock) > "main" prio=1 tid=0x0805cac8 nid=0x2a68 in Object.wait() [0xbfffc000..0xbfffd1a8] > at java.lang.Object.wait(Native Method) > - waiting on <0x36417cd0> (a [Z) > at java.lang.Object.wait(Object.java:474) > at org.apache.activemq.console.command.StartCommand.waitForShutdown(StartCommand.java:106) > - locked <0x36417cd0> (a [Z) > at org.apache.activemq.console.command.StartCommand.runTask(StartCommand.java:67) > at org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:49) > at org.apache.activemq.console.command.ShellCommand.runTask(ShellCommand.java:81) > at org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:49) > at org.apache.activemq.console.command.ShellCommand.main(ShellCommand.java:45) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:585) > at org.apache.activemq.console.Main.runTaskClass(Main.java:135) > at org.apache.activemq.console.Main.main(Main.java:67) > "VM Thread" prio=1 tid=0x08096ae8 nid=0x2a70 runnable > "VM Periodic Task Thread" prio=1 tid=0x080a4b70 nid=0x2a76 waiting on condition > Found one Java-level deadlock: > ============================= > "ActiveMQ Failover Worker: 5862378": > waiting to lock monitor 0x0809c22c (object 0x36579610, a java.lang.Object), > which is held by "Thread-7" > "Thread-7": > waiting to lock monitor 0x0809c2ac (object 0x365793b0, a java.lang.Object), > which is held by "ActiveMQ Failover Worker: 5862378" > Java stack information for the threads listed above: > =================================================== > "ActiveMQ Failover Worker: 5862378": > at org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:43) > - waiting to lock <0x36579610> (a java.lang.Object) > at org.apache.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelator.java:60) > at org.apache.activemq.network.DemandForwardingBridgeSupport$2.transportResumed(DemandForwardingBridgeSupport.java:164) > - locked <0x3654b290> (a org.apache.activemq.network.DemandForwardingBridgeSupport$2) > at org.apache.activemq.transport.TransportFilter.transportResumed(TransportFilter.java:102) > at org.apache.activemq.transport.TransportFilter.transportResumed(TransportFilter.java:102) > at org.apache.activemq.transport.failover.FailoverTransport$2.iterate(FailoverTransport.java:175) > - locked <0x365793b0> (a java.lang.Object) > at org.apache.activemq.thread.DedicatedTaskRunner.runTask(DedicatedTaskRunner.java:87) > at org.apache.activemq.thread.DedicatedTaskRunner.access$000(DedicatedTaskRunner.java:24) > at org.apache.activemq.thread.DedicatedTaskRunner$1.run(DedicatedTaskRunner.java:38) > "Thread-7": > at org.apache.activemq.transport.failover.FailoverTransport.oneway(FailoverTransport.java:335) > - waiting to lock <0x365793b0> (a java.lang.Object) > at org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:44) > - locked <0x36579610> (a java.lang.Object) > at org.apache.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelator.java:60) > at org.apache.activemq.network.DemandForwardingBridgeSupport.startRemoteBridge(DemandForwardingBridgeSupport.java:235) > at org.apache.activemq.network.DemandForwardingBridgeSupport$4.run(DemandForwardingBridgeSupport.java:197) > Found 1 deadlock. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.