Hello,

I am working on a client/server product where the server component has an embedded ActiveMQ broker. After upgrading the ActiveMQ from 5.3.1 to 5.4.2 we always hit a server side problem where the call to create a consumer gets stuck in waiting for a response:

"MonitorServer-RoundRobinMessageHandlerThread-1" prio=10 tid=0x00007fd0d8696800 nid=0x7b29 waiting on condition [0x00007fcecd0cf000]
   java.lang.Thread.State: WAITING (parking)
    at sun.misc.Unsafe.park(Native Method)
    - parking to wait for  <0x00007fcfe489edb0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
    at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:317)
    at org.apache.activemq.transport.FutureResponse.getResult(FutureResponse.java:40)
    at org.apache.activemq.transport.ResponseCorrelator.request(ResponseCorrelator.java:87)
    at org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1276)
    at org.apache.activemq.ActiveMQSession.syncSendPacket(ActiveMQSession.java:1874)
    at org.apache.activemq.ActiveMQMessageConsumer.<init>(ActiveMQMessageConsumer.java:254)
    at org.apache.activemq.ActiveMQSession.createConsumer(ActiveMQSession.java:1116)
    at org.apache.activemq.ActiveMQSession.createConsumer(ActiveMQSession.java:1060)
    at org.apache.activemq.ActiveMQSession.createConsumer(ActiveMQSession.java:973)
    at org.apache.activemq.ActiveMQSession.createConsumer(ActiveMQSession.java:946)
    at com.mycompany.monitor.server.MonitorServer.handleMessagesFromAllConsumers(MonitorServer.java:167)
    at com.mycompany.monitor.server.MonitorServer.receiveMessages(MonitorServer.java:122)
    at com.mycompany.monitor.server.MonitorServer.access$000(MonitorServer.java:31)
    at com.mycompany.monitor.server.MonitorServer$1.run(MonitorServer.java:84)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:619)

To give you some background on how things are organized: the clients create queues and send messages to them. The server listens for created destinations it creates and caches a consumer once a new destination is found and then tries to consume the messages. The above thread dump shows the code getting stuck in the createConsumer() part. The problem outlined here is only seen in production, so we have limited time to debug/investigate. All our attempts to replicate the problem on a "non-production" machine have failed so far (we tried simulating a large number of clients, messages per seconds, using client/server disconnects etc). 

In terms of environment, the clients run on separate machines that are either Fedora 13 or Windows, while the server is a Fedora 13, running Sun JDK 1.6.0_20.

Does anyone know what could be the problem and how to find the root cause?

Thank you!

Best Regards,

Horace