activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Martin C." <mart...@gmx.at>
Subject Randomly hanging consumers
Date Tue, 02 Aug 2011 06:50:02 GMT
Hi,

I have a problem in several of my deployments using ActiveMQ 5.5.0,
where on a certain connection messages suddenly stop being consumed. I
have not yet been able to determine a pattern when this happens, I
suppose it is some kind of race condition.

The setup is a modified Spring DefaultMessageListenerContainer which
was modified to process 10 messages within a single XA transaction.
There are two concurrent consumers sharing a single connection, both
consumers using XA, using Atomikos as transaction manager. It is a
plain SSL conneciton, no failover transport. Broker is on localhost in
this case, so no network issues should be involved. Prefetch is set to
1 as we are using Spring.

After several hundred thousand messages, the connection suddenly gets
stuck and both consumer keep hanging without receiving any new
messages. All other connections (with consumers to other queues) keep
on going without problems.

Using JMX I stopped the connection of the hanging consumers, this
caused them to fail their current call to receive and after
re-connecting all pending messages get consumed.

Taking a thread-dump of the hanging consumers shows that they seem to
be waiting for a dequeue.

"batchedListenerContainer-291" prio=10 tid=0x0000000002915800
nid=0x2cda in Object.wait() [0x00007fc0d46a4000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00000000ec4bc828> (a java.lang.Object)
        at org.apache.activemq.SimplePriorityMessageDispatchChannel.dequeue(SimplePriorityMessageDispatchChannel.java:89)
        - locked <0x00000000ec4bc828> (a java.lang.Object)
        at org.apache.activemq.ActiveMQMessageConsumer.dequeue(ActiveMQMessageConsumer.java:452)
        at org.apache.activemq.ActiveMQMessageConsumer.receive(ActiveMQMessageConsumer.java:577)
        at com.atomikos.jms.AtomikosJmsMessageConsumerProxy.receive(AtomikosJmsMessageConsumerProxy.java:74)
        at com.atomikos.jms.AtomikosJmsMessageConsumerProxy.receive(AtomikosJmsMessageConsumerProxy.java:140)
        at invalid.company.jms.BatchedMessageListenerContainer.doReceiveAndExecute(BatchedMessageListenerContainer.java:94)
        at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:243)
        at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1058)
        at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:952)
        at java.lang.Thread.run(Thread.java:662)

"batchedListenerContainer-290" prio=10 tid=0x00000000036ce800
nid=0x2ca3 in Object.wait() [0x00007fc0cc5c4000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00000000ecdf3348> (a java.lang.Object)
        at org.apache.activemq.SimplePriorityMessageDispatchChannel.dequeue(SimplePriorityMessageDispatchChannel.java:89)
        - locked <0x00000000ecdf3348> (a java.lang.Object)
        at org.apache.activemq.ActiveMQMessageConsumer.dequeue(ActiveMQMessageConsumer.java:452)
        at org.apache.activemq.ActiveMQMessageConsumer.receive(ActiveMQMessageConsumer.java:577)
        at com.atomikos.jms.AtomikosJmsMessageConsumerProxy.receive(AtomikosJmsMessageConsumerProxy.java:74)
        at com.atomikos.jms.AtomikosJmsMessageConsumerProxy.receive(AtomikosJmsMessageConsumerProxy.java:140)
        at invalid.company.jms.BatchedMessageListenerContainer.doReceiveAndExecute(BatchedMessageListenerContainer.java:94)
        at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:243)
        at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1058)
        at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:952)
        at java.lang.Thread.run(Thread.java:662)

Is this a known issue, should I report this as a bug to Jira, has
anybody got an idea what I could do about this, as this is a
production issue and keeps causing me a lot of support incidents?

Thanks in advance for any help!

Best regards,
Martin

Mime
View raw message