activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From brian <brian.a...@conceptual.ca>
Subject confused about Usage Manager Memory Limit reached
Date Tue, 09 Dec 2014 01:46:47 GMT
Using ActiveMQ 5.9.1

Our setup: A Spring/JMS producer places many (sometimes quite large) 
messages on one queue (hf.request), which are being picked up by a PHP 
Stomp client. PHP processes the data and sends a message to a second 
queue (hf.response) to let it know how things went.

Occasionally, we're seeing the PHP producer being blocked.

Starting with both IDs known ...

php (stomp): ID:pan-51744-1418069977147-2:2
tcp://127.0.0.1:34916

java (openwire): ID:pan-36709-1418086070781-1:1
tcp://127.0.0.1:46827


2014-12-08 20:17:26,535 | INFO  | 
Usage(default:memory:queue://hf.response:memory) percentUsage=0%, 
usage=0, limit=720791142, 
percentUsageMinDelta=1%;Parent:Usage(default:memory) percentUsage=104%, 
usage=752535801, limit=720791142, percentUsageMinDelta=1%: Usage Manager 
Memory Limit reached. Producer (ID:pan-51744-1418069977147-2:2:-1:1) 
stopped to prevent flooding queue://hf.response. See 
http://activemq.apache.org/producer-flow-control.html for more info 
(blocking for: 1352s) | org.apache.activemq.broker.region.Queue | 
ActiveMQ Transport: tcp:///127.0.0.1:34916@61613


The log message is confusing as it appears to be telling me that the 
response queue is being flooded. However, the amount of data sent to 
response is trivial:

2014-12-08 19:55:01 Info: Array
(
     [synchronizationId] => 6676ab36-f6cf-402d-9045-b58e210fcd3e
     [type] => INVENTORY_ITEM
     [command] => CREATE
     [status] => OK
     [statusMessage] =>
)


... compared to request, which can amount to several MB per msg. And it 
can't possibly be that PHP is hitting the queue too fast as it has a 
fair bit of data to process before it sends to hf.response.

The browser client shows that all messages enqueued to response have 
been successfully dequeued, but there are several hundred pending in 
hf.request.

Can anyone tell me what might be going on here? If anything, i'd expect 
the java producer to be blocked to prevent hf.request from being 
flooded. That is, i would have thought that the place where we'd run 
into trouble would be constantPendingMessageLimitStrategy. Here's the 
config for that:

<pendingMessageLimitStrategy>
   <constantPendingMessageLimitStrategy limit="1000"/>
</pendingMessageLimitStrategy>

System usage is:

<systemUsage>
   <systemUsage>
       <memoryUsage>
           <memoryUsage percentOfJvmHeap="70" />
       </memoryUsage>
       <storeUsage>
           <storeUsage limit="100 gb"/>
       </storeUsage>
       <tempUsage>
           <tempUsage limit="50 gb"/>
       </tempUsage>
   </systemUsage>
</systemUsage>


The only thing i can think of is that the log message, "stopped to 
prevent flooding queue://hahnfarms.response" is getting its queue name 
from the hf.request messages' reply-to headers, but that the real 
problem is with hf.request. But then, i'd presume that it would be the 
java client that would be blocked.



Mime
View raw message