Mailing-List: contact activemq-users-help@geronimo.apache.org; run by ezmlm
Precedence: bulk
Reply-To: activemq-users@geronimo.apache.org
Received-SPF: pass (herse.apache.org: domain of james.strachan@gmail.com
 designates 66.249.92.173 as permitted sender)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws;
        s=beta; d=gmail.com;
        h=received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references;
        b=PFUSvLXabldNGg5zrDKiSQjd4nBzWiKFOAJCHjS069Gx3Hcu214XA3BJqpTTRkgFxWEC1QGTJAXM84ElxBLboVN26ZwE6vPRa0r3IPulR3G6+M6S5L5dq159wxNxBNUBzfS2rBJvCOF8n/z9R7lo+EXytw8cjKBTDOqcXw38jts=
Message-ID: <ec6e67fd0612140735v4f008194w3282ab1bf8c78efd@mail.gmail.com>
Date: Thu, 14 Dec 2006 15:35:30 +0000
From: "James Strachan" <james.strachan@gmail.com>
To: activemq-users@geronimo.apache.org
Subject: Re: Client deadlock in AMQ 4.1.0
In-Reply-To: <7828363.post@talk.nabble.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
References: <7793886.post@talk.nabble.com> <7828363.post@talk.nabble.com>

Thanks for the great stack trace! That really helps figuring this out.

I've just committed a fix to trunk which I think should fix this issue
- please could you retest with trunk and see if that fixes your issue?


On 12/12/06, Danielius Jurna <danielius@elitnet.lt> wrote:
>
> There's jira issue for this bug:
> https://issues.apache.org/activemq/browse/AMQ-1093
>
>
> Danielius Jurna wrote:
> >
> > In 4.1.0 there's deadlock on connection failover. There is the scenario:
> > 1. Client consumes message using message listener
> > 2. Conection is lost
> > 3. Client sends message to another queue from messagle listener and waits
> > untill connection is restored.
> > 4. Reconnect task blocks on reconnecting.
> >
> > This bug is new to 4.1.0. The problem is in
> > ActiveMQMessageConsumre.dispatch . There is new lock on
> > unconsumedMessages.getMutex() . So the reconnect task cannot invoke
> > ActiveMQMessageConsumre.clearMessagesInProgress(), because lock is
> > acquired by message listener, which waits untill message is send (untill
> > connection is resumed). Here is stack traces:
> >
> > "ActiveMQ Session Task" daemon prio=1 tid=0x0000002b27774260 nid=0x4778 in
> > Object.wait() [0x0000000040ef3000..0x0000000040ef4db0]
> >       at java.lang.Object.wait(Native Method)
> >       - waiting on <0x0000002b0020a7c8> (a
> > edu.emory.mathcs.backport.java.util.concurrent.locks.CondVar)
> >       at java.lang.Object.wait(Object.java:474)
> >       at
> > edu.emory.mathcs.backport.java.util.concurrent.locks.CondVar.await(CondVar.java:75)
> >       - locked <0x0000002b0020a7c8> (a
> > edu.emory.mathcs.backport.java.util.concurrent.locks.CondVar)
> >       at
> > edu.emory.mathcs.backport.java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:318)
> >       at
> > org.apache.activemq.transport.FutureResponse.getResult(FutureResponse.java:42)
> >       at
> > org.apache.activemq.transport.ResponseCorrelator.request(ResponseCorrelator.java:75)
> >       at
> > org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1171)
> >       at org.apache.activemq.ActiveMQSession.send(ActiveMQSession.java:1548)
> >       at
> > org.apache.activemq.ActiveMQMessageProducer.send(ActiveMQMessageProducer.java:465)
> >       at org.apache.activemq.pool.PooledProducer.send(PooledProducer.java:75)
> >       - locked <0x0000002b173fa480> (a
> > org.apache.activemq.ActiveMQMessageProducer)
> >       at org.apache.activemq.pool.PooledProducer.send(PooledProducer.java:60)
> >       at org.springframework.jms.core.JmsTemplate.doSend(JmsTemplate.java:537)
> >       at org.springframework.jms.core.JmsTemplate.doSend(JmsTemplate.java:513)
> >       at
> > org.springframework.jms.core.JmsTemplate$2.doInJms(JmsTemplate.java:479)
> >       at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:430)
> >       at org.springframework.jms.core.JmsTemplate.send(JmsTemplate.java:477)
> >       at
> > lt.elitnet.dbp.das.impl.storage.HI2StorageImpl.storeHI2Message(HI2StorageImpl.java:57)
> >       at
> > lt.elitnet.dbp.das.impl.hi2.HI2PersistanceBase.saveIRIContent(HI2PersistanceBase.java:77)
> >       at sun.reflect.GeneratedMethodAccessor185.invoke(Unknown Source)
> >       at
> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> >       at java.lang.reflect.Method.invoke(Method.java:585)
> >       at
> > org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:318)
> >       at
> > org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:203)
> >       at
> > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:162)
> >       at
> > org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:107)
> >       at
> > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
> >       at
> > org.springframework.orm.hibernate3.HibernateInterceptor.invoke(HibernateInterceptor.java:104)
> >       at
> > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
> >       at
> > lt.elitnet.dbp.das.impl.alarming.DataBaseConnectionAlarmsPublisher.invoke(DataBaseConnectionAlarmsPublisher.java:59)
> >       at
> > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
> >       at
> > org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:209)
> >       at $Proxy24.saveIRIContent(Unknown Source)
> >       at
> > lt.elitnet.dbp.das.impl.hi2.HI2MessageListener.onMessage(HI2MessageListener.java:40)
> >       at
> > lt.elitnet.dbp.commons.messaging.TextMessageListener.deliverMessage(TextMessageListener.java:84)
> >       at
> > lt.elitnet.dbp.commons.messaging.TextMessageListener.onMessage(TextMessageListener.java:50)
> >       at
> > org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:854)
> >       at
> > org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:793)
> >       at
> > org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:763)
> >       at
> > org.springframework.jms.listener.AbstractMessageListenerContainer.executeListener(AbstractMessageListenerContainer.java:743)
> >       at
> > org.springframework.jms.listener.SimpleMessageListenerContainer$2.onMessage(SimpleMessageListenerContainer.java:190)
> >       at
> > org.apache.activemq.ActiveMQMessageConsumer.dispatch(ActiveMQMessageConsumer.java:840)
> >       - locked <0x0000002b1771ddc0> (a java.lang.Object)
> >       at
> > org.apache.activemq.ActiveMQSessionExecutor.dispatch(ActiveMQSessionExecutor.java:96)
> >       at
> > org.apache.activemq.ActiveMQSessionExecutor.iterate(ActiveMQSessionExecutor.java:165)
> >       at
> > org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:111)
> >       at
> > org.apache.activemq.thread.PooledTaskRunner.access$100(PooledTaskRunner.java:26)
> >       at
> > org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:44)
> >       at
> > edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
> >       at
> > edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
> >       at java.lang.Thread.run(Thread.java:595)
> >
> > "ActiveMQ Transport: tcp:///192.168.2.55:61616" prio=1
> > tid=0x0000002b2e814660 nid=0x4773 waiting for monitor entry
> > [0x0000000040e82000..0x0000000040e82c30]
> >       at
> > org.apache.activemq.MessageDispatchChannel.clear(MessageDispatchChannel.java:135)
> >       - waiting to lock <0x0000002b1771ddc0> (a java.lang.Object)
> >       at
> > org.apache.activemq.ActiveMQMessageConsumer.clearMessagesInProgress(ActiveMQMessageConsumer.java:546)
> >       at
> > org.apache.activemq.ActiveMQSession.clearMessagesInProgress(ActiveMQSession.java:524)
> >       at
> > org.apache.activemq.ActiveMQConnection.transportInterupted(ActiveMQConnection.java:1547)
> >       at
> > org.apache.activemq.transport.TransportFilter.transportInterupted(TransportFilter.java:100)
> >       at
> > org.apache.activemq.transport.TransportFilter.transportInterupted(TransportFilter.java:100)
> >       at
> > org.apache.activemq.transport.failover.FailoverTransport.handleTransportFailure(FailoverTransport.java:236)
> >       at
> > org.apache.activemq.transport.failover.FailoverTransport.access$300(FailoverTransport.java:55)
> >       at
> > org.apache.activemq.transport.failover.FailoverTransport$1.onException(FailoverTransport.java:120)
> >       at
> > org.apache.activemq.transport.TransportFilter.onException(TransportFilter.java:96)
> >       at
> > org.apache.activemq.transport.WireFormatNegotiator.onException(WireFormatNegotiator.java:147)
> >       at
> > org.apache.activemq.transport.InactivityMonitor.onException(InactivityMonitor.java:150)
> >       at
> > org.apache.activemq.transport.TransportSupport.onException(TransportSupport.java:97)
> >       at
> > org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:150)
> >       at java.lang.Thread.run(Thread.java:595)
> >
>
> --
> View this message in context: http://www.nabble.com/Client-deadlock-in-AMQ-4.1.0-tf2793676.html#a7828363
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>


-- 

James
-------
http://radio.weblogs.com/0112098/