From activemq-users-return-6196-apmail-geronimo-activemq-users-archive=geronimo.apache.org@geronimo.apache.org Mon Dec 11 12:36:58 2006 Return-Path: Delivered-To: apmail-geronimo-activemq-users-archive@www.apache.org Received: (qmail 96392 invoked from network); 11 Dec 2006 12:36:56 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 11 Dec 2006 12:36:56 -0000 Received: (qmail 77663 invoked by uid 500); 11 Dec 2006 12:37:03 -0000 Delivered-To: apmail-geronimo-activemq-users-archive@geronimo.apache.org Received: (qmail 77651 invoked by uid 500); 11 Dec 2006 12:37:03 -0000 Mailing-List: contact activemq-users-help@geronimo.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: activemq-users@geronimo.apache.org Delivered-To: mailing list activemq-users@geronimo.apache.org Received: (qmail 77642 invoked by uid 99); 11 Dec 2006 12:37:03 -0000 Received: from herse.apache.org (HELO herse.apache.org) (140.211.11.133) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 11 Dec 2006 04:37:03 -0800 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_HELO_PASS,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (herse.apache.org: domain of lists@nabble.com designates 72.21.53.35 as permitted sender) Received: from [72.21.53.35] (HELO talk.nabble.com) (72.21.53.35) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 11 Dec 2006 04:36:51 -0800 Received: from [72.21.53.38] (helo=jubjub.nabble.com) by talk.nabble.com with esmtp (Exim 4.50) id 1GtkOU-0002wJ-Jb for activemq-users@geronimo.apache.org; Mon, 11 Dec 2006 04:36:30 -0800 Message-ID: <7793886.post@talk.nabble.com> Date: Mon, 11 Dec 2006 04:36:30 -0800 (PST) From: Danielius Jurna To: activemq-users@geronimo.apache.org Subject: Client deadlock in AMQ 4.1.0 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Nabble-From: danielius@elitnet.lt X-Virus-Checked: Checked by ClamAV on apache.org In 4.1.0 there's deadlock on connection failover. There is the scenario: 1. Client consumes message using message listener 2. Conection is lost 3. Client sends message to another queue from messagle listener and waits untill connection is restored. 4. Reconnect task blocks on reconnecting. This bug is new to 4.1.0. The problem is in ActiveMQMessageConsumre.dispatch . There is new lock on unconsumedMessages.getMutex() . So the reconnect task cannot invoke ActiveMQMessageConsumre.clearMessagesInProgress(), because lock is acquired by message listener, which waits untill message is send (untill connection is resumed). Here is stack traces: "ActiveMQ Session Task" daemon prio=1 tid=0x0000002b27774260 nid=0x4778 in Object.wait() [0x0000000040ef3000..0x0000000040ef4db0] at java.lang.Object.wait(Native Method) - waiting on <0x0000002b0020a7c8> (a edu.emory.mathcs.backport.java.util.concurrent.locks.CondVar) at java.lang.Object.wait(Object.java:474) at edu.emory.mathcs.backport.java.util.concurrent.locks.CondVar.await(CondVar.java:75) - locked <0x0000002b0020a7c8> (a edu.emory.mathcs.backport.java.util.concurrent.locks.CondVar) at edu.emory.mathcs.backport.java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:318) at org.apache.activemq.transport.FutureResponse.getResult(FutureResponse.java:42) at org.apache.activemq.transport.ResponseCorrelator.request(ResponseCorrelator.java:75) at org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1171) at org.apache.activemq.ActiveMQSession.send(ActiveMQSession.java:1548) at org.apache.activemq.ActiveMQMessageProducer.send(ActiveMQMessageProducer.java:465) at org.apache.activemq.pool.PooledProducer.send(PooledProducer.java:75) - locked <0x0000002b173fa480> (a org.apache.activemq.ActiveMQMessageProducer) at org.apache.activemq.pool.PooledProducer.send(PooledProducer.java:60) at org.springframework.jms.core.JmsTemplate.doSend(JmsTemplate.java:537) at org.springframework.jms.core.JmsTemplate.doSend(JmsTemplate.java:513) at org.springframework.jms.core.JmsTemplate$2.doInJms(JmsTemplate.java:479) at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:430) at org.springframework.jms.core.JmsTemplate.send(JmsTemplate.java:477) at lt.elitnet.dbp.das.impl.storage.HI2StorageImpl.storeHI2Message(HI2StorageImpl.java:57) at lt.elitnet.dbp.das.impl.hi2.HI2PersistanceBase.saveIRIContent(HI2PersistanceBase.java:77) at sun.reflect.GeneratedMethodAccessor185.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:585) at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:318) at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:203) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:162) at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:107) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185) at org.springframework.orm.hibernate3.HibernateInterceptor.invoke(HibernateInterceptor.java:104) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185) at lt.elitnet.dbp.das.impl.alarming.DataBaseConnectionAlarmsPublisher.invoke(DataBaseConnectionAlarmsPublisher.java:59) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:209) at $Proxy24.saveIRIContent(Unknown Source) at lt.elitnet.dbp.das.impl.hi2.HI2MessageListener.onMessage(HI2MessageListener.java:40) at lt.elitnet.dbp.commons.messaging.TextMessageListener.deliverMessage(TextMessageListener.java:84) at lt.elitnet.dbp.commons.messaging.TextMessageListener.onMessage(TextMessageListener.java:50) at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:854) at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:793) at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:763) at org.springframework.jms.listener.AbstractMessageListenerContainer.executeListener(AbstractMessageListenerContainer.java:743) at org.springframework.jms.listener.SimpleMessageListenerContainer$2.onMessage(SimpleMessageListenerContainer.java:190) at org.apache.activemq.ActiveMQMessageConsumer.dispatch(ActiveMQMessageConsumer.java:840) - locked <0x0000002b1771ddc0> (a java.lang.Object) at org.apache.activemq.ActiveMQSessionExecutor.dispatch(ActiveMQSessionExecutor.java:96) at org.apache.activemq.ActiveMQSessionExecutor.iterate(ActiveMQSessionExecutor.java:165) at org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:111) at org.apache.activemq.thread.PooledTaskRunner.access$100(PooledTaskRunner.java:26) at org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:44) at edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650) at edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675) at java.lang.Thread.run(Thread.java:595) "ActiveMQ Transport: tcp:///192.168.2.55:61616" prio=1 tid=0x0000002b2e814660 nid=0x4773 waiting for monitor entry [0x0000000040e82000..0x0000000040e82c30] at org.apache.activemq.MessageDispatchChannel.clear(MessageDispatchChannel.java:135) - waiting to lock <0x0000002b1771ddc0> (a java.lang.Object) at org.apache.activemq.ActiveMQMessageConsumer.clearMessagesInProgress(ActiveMQMessageConsumer.java:546) at org.apache.activemq.ActiveMQSession.clearMessagesInProgress(ActiveMQSession.java:524) at org.apache.activemq.ActiveMQConnection.transportInterupted(ActiveMQConnection.java:1547) at org.apache.activemq.transport.TransportFilter.transportInterupted(TransportFilter.java:100) at org.apache.activemq.transport.TransportFilter.transportInterupted(TransportFilter.java:100) at org.apache.activemq.transport.failover.FailoverTransport.handleTransportFailure(FailoverTransport.java:236) at org.apache.activemq.transport.failover.FailoverTransport.access$300(FailoverTransport.java:55) at org.apache.activemq.transport.failover.FailoverTransport$1.onException(FailoverTransport.java:120) at org.apache.activemq.transport.TransportFilter.onException(TransportFilter.java:96) at org.apache.activemq.transport.WireFormatNegotiator.onException(WireFormatNegotiator.java:147) at org.apache.activemq.transport.InactivityMonitor.onException(InactivityMonitor.java:150) at org.apache.activemq.transport.TransportSupport.onException(TransportSupport.java:97) at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:150) at java.lang.Thread.run(Thread.java:595) -- View this message in context: http://www.nabble.com/Client-deadlock-in-AMQ-4.1.0-tf2793676.html#a7793886 Sent from the ActiveMQ - User mailing list archive at Nabble.com.