activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ishitori <Ishit...@bk.ru>
Subject ActiveMQ.NMS consumer hangs after sometime
Date Tue, 07 Feb 2012 13:17:45 GMT
I am using 2 ActiveMQ brokers 5.5.0 with failover on Windows. I have a C#
windows service which connects to the broker queue via NMS. Windows service
creates 100 threads and each thread has its own consumer. I use
AutoAcknowledge mode. I run 4 instances of the windows service on separate
machines. PrefetchSize is set to 1 and I use synchronous receive method with
small (10 ms) wait time.

I also have the following setup on brokers for queues:

<policyEntry queue=">" producerFlowControl="false" memoryLimit="1mb">

After working for some period of time (usually a day or two) all consumers
stop receiving any messages, but I can see messages in brokers. Usually all
windows service instances stop receiving messages simultaneously, but not
every time. If I restart services then they start processing messages.

I've created a dump file and I found a lot of threads are waiting for a
mutex to be released. Here is the stack trace:

System.Threading.Monitor.Wait(System.Object, System.TimeSpan)
Apache.NMS.ActiveMQ.Util.SimplePriorityMessageDispatchChannel.Dequeue(System.TimeSpan)
Apache.NMS.ActiveMQ.MessageConsumer.Dequeue(System.TimeSpan)
Apache.NMS.ActiveMQ.MessageConsumer.Receive(System.TimeSpan)

I also have a lot of DedicatedTaskRunners waiting for something:

System.Threading.Monitor.Wait(System.Object)
Apache.NMS.ActiveMQ.Threads.DedicatedTaskRunner.Run()
System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext,
System.Threading.ContextCallback, System.Object, Boolean)
System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext,
System.Threading.ContextCallback, System.Object)
System.Threading.ThreadHelper.ThreadStart()

I've found a few issues with the same problem, but I can't find my case:

1)
http://activemq.2283324.n4.nabble.com/Client-threads-hang-if-ActiveMQ-dies-td2363080.html
- I don't use transacted messages anymore, neither prefetchSize=0. I don't
use ReceiveNoWait as well (though I could move waiting logic to my
application), because it seems to cause some problems.

2)
http://activemq.2283324.n4.nabble.com/Consumer-Receive-hangs-td2364683.html
- I set producerFlowControl to false, and afair the broker could use as much
memory as the server has.

I also found an issue which describes the same problem and is marked as
resolved - https://issues.apache.org/jira/browse/AMQ-2446

What else could cause such a problem?

--
View this message in context: http://activemq.2283324.n4.nabble.com/ActiveMQ-NMS-consumer-hangs-after-sometime-tp4364689p4364689.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Mime
View raw message