activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jason Rosenberg <jrosenb...@shopzilla.com>
Subject Deadlock condition, producer in waitForSpace, consumer slow, etc.
Date Tue, 15 May 2007 17:26:13 GMT

Hi all, I'm just using ActiveMQ for the first time.  I've encountered a
deadlock condition, that appears rooted in the ActiveMQ internals.  I'm not
sure if this is a known problem or not.  I've been able to work around the
issue, but thought I'd forward this on, to see if it warrants filing a bug,
or if it's a 'feature'....

I am using version 4.1.1, and am running it within tomcat 5.5.23, using the
vm://localhost broker, and am using non-persistent messaging, with
AUTO_ACKNOWLEDGE.  I am using queues, and the consumers are triggered using
the setMessageListener api.

Essentially what seems to be happening is that the producer generates more
messages than the consumer can keep up with, and eventually appears to cause
the message queue to run out of space.   What I would expect in this case,
is that the consumer would still be able to process messages, at it's own
slow pace....and as it processes messages, space would open up in the queue,
and the producer would be able to slowly continue submitting more
messages....

I realize normally this is not an ideal use of messaging, e.g. you don't
want to generate more messages than you can possibly consume,
etc....However, I would expect that it would still make progress, etc..

I am instead seeing a simple deadlock condition, the producer locks up while
waiting for space, and locks a monitor that the consumer appears to be
waiting on....I've included a partial thread dump below, with the 2 relevant
threads:

"ActiveMQ Session Task" daemon prio=7 tid=0x0055be30 nid=0x18ac000 waiting
for monitor entry [0xb1820000..0xb1820d10]
        at
org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:46)
        - waiting to lock <0x2704bef8> (a java.lang.Object)
        at
org.apache.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelator.java:60)
        at
org.apache.activemq.ActiveMQConnection.asyncSendPacket(ActiveMQConnection.java:1155)
        at
org.apache.activemq.ActiveMQSession.asyncSendPacket(ActiveMQSession.java:1651)
        at
org.apache.activemq.ActiveMQMessageConsumer.afterMessageIsConsumed(ActiveMQMessageConsumer.java:679)
        at
org.apache.activemq.ActiveMQMessageConsumer.dispatch(ActiveMQMessageConsumer.java:855)
        at
org.apache.activemq.ActiveMQSessionExecutor.dispatch(ActiveMQSessionExecutor.java:99)
        at
org.apache.activemq.ActiveMQSessionExecutor.iterate(ActiveMQSessionExecutor.java:166)
        at
org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:117)
        at
org.apache.activemq.thread.PooledTaskRunner.access$100(PooledTaskRunner.java:26)
        at
org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:44)
        at
edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
        at
edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
        at java.lang.Thread.run(Thread.java:613)



"http-8080-Processor25" daemon prio=5 tid=0x00540ed0 nid=0x1902800 in
Object.wait() [0xb27bf000..0xb27bfd10]
        at java.lang.Object.wait(Native Method)
        - waiting on <0x2702d7c8> (a java.lang.Object)
        at
org.apache.activemq.memory.UsageManager.waitForSpace(UsageManager.java:105)
        - locked <0x2702d7c8> (a java.lang.Object)
        at
org.apache.activemq.memory.UsageManager.waitForSpace(UsageManager.java:100)
        at org.apache.activemq.broker.region.Queue.send(Queue.java:318)
        at
org.apache.activemq.broker.region.AbstractRegion.send(AbstractRegion.java:292)
        at
org.apache.activemq.broker.region.RegionBroker.send(RegionBroker.java:382)
        at
org.apache.activemq.broker.TransactionBroker.send(TransactionBroker.java:193)
        at
org.apache.activemq.broker.BrokerFilter.send(BrokerFilter.java:126)
        at
org.apache.activemq.broker.CompositeDestinationBroker.send(CompositeDestinationBroker.java:98)
        at
org.apache.activemq.broker.MutableBrokerFilter.send(MutableBrokerFilter.java:136)
        at
org.apache.activemq.broker.TransportConnection.processMessage(TransportConnection.java:498)
        at
org.apache.activemq.command.ActiveMQMessage.visit(ActiveMQMessage.java:604)
        at
org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:294)
        at
org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:185)
        at
org.apache.activemq.transport.ResponseCorrelator.onCommand(ResponseCorrelator.java:95)
        at
org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:65)
        at
org.apache.activemq.transport.vm.VMTransport.syncOneWay(VMTransport.java:96)
        at
org.apache.activemq.transport.vm.VMTransport.oneway(VMTransport.java:83)
        at
org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:47)
        - locked <0x2704bef8> (a java.lang.Object)
        at
org.apache.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelator.java:60)
        at
org.apache.activemq.ActiveMQConnection.asyncSendPacket(ActiveMQConnection.java:1155)
        at
org.apache.activemq.ActiveMQSession.send(ActiveMQSession.java:1549)
        at
org.apache.activemq.ActiveMQMessageProducer.send(ActiveMQMessageProducer.java:465)
        at
org.apache.activemq.ActiveMQMessageProducer.send(ActiveMQMessageProducer.java:356)
        at
com.shopzilla.utils.NonPersistentLocalMessaging.sendMessage(NonPersistentLocalMessaging.java:208)
        at
com.shopzilla.its.ws.ITSSessionizationFileCollector._sendDataListMessage(ITSSessionizationFileCollector.java:140)
        at
com.shopzilla.its.ws.ITSSessionizationFileCollector.doPost(ITSSessionizationFileCollector.java:101)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:710)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
        at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)
        at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
        at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:210)
        at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:174)
        at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
        at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
        at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
        at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:151)
        at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:870)
        at
org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665)
        at
org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528)
        at
org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81)
        at
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:685)
        at java.lang.Thread.run(Thread.java:613)


Thanks,

Jason
-- 
View this message in context: http://www.nabble.com/Deadlock-condition%2C-producer-in-waitForSpace%2C-consumer-slow%2C-etc.-tf3759923s2354.html#a10627274
Sent from the ActiveMQ - User mailing list archive at Nabble.com.


Mime
View raw message