activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Elliot Barlas <elliotbar...@gmail.com>
Subject Why do persistent messages exhaust memory and lock out consumers?
Date Thu, 11 Jun 2009 00:48:06 GMT

Scenario:
-Minimal AMQ message broker with persistence enabled and default usage
values of 64 MB memory limit, 10 GB temp store limit, and 100 GB persistent
store limit
-AMQ producer client sending 100 KB persistent messages to queue X in a loop
with a short sleep (roughly 10 msgs/second)

Observations:
-MemoryPercentUsage reaches 70% after a few minutes and remains there
-StorePercentUsage remains at 0 (since persistent store limit is so high)
-Disk usage under persistent data directory increases as expected


Why is the memory percent usage increasing so rapidly?  Shouldn't messages
go directly to the persistent store?  If memory is used, shouldn't it be
flushed once it reaches a certain threshold?


Scenario continued:
-A second AMQ producer client sending 100 KB persistent messages to queue Y
in a loop
-AMQ consumer client consuming messages from queue Y in a loop

Observations continued:
-MemoryPercentUsage remains at 70%
-StorePercentUsage remains at 0
-Disk usage under persistent data directory continues to increase
-NO messages dequeued from queue Y!


Shockingly, messages are NOT consumed from queue Y in the scenario described
above.  Once the 70% memory usage threshold is reached, messages cannot be
consumed from queue Y.  After further testing, I have seen the messages can
be consumed from queue X, which decreased the memory percent used, and
restores the system (messages can then be consumed from other queues).

Why is this happening?  Has anyone else seen this?

The same test with non-persistent messages is not problematic.  The memory
percent usage increases, but once it reaches 70% the messages are flushed to
the temp store and the percent usage is reduce to 0%

Thanks,
Elliot
-- 
View this message in context: http://www.nabble.com/Why-do-persistent-messages-exhaust-memory-and-lock-out-consumers--tp23973571p23973571.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.


Mime
View raw message