activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bruce Snyder <bruce.sny...@gmail.com>
Subject Re: Global memory limit for all queues / fileQueueCursor question
Date Tue, 17 Aug 2010 18:48:17 GMT
On Tue, Aug 17, 2010 at 10:52 AM, Martin C. <martinc@gmx.at> wrote:
>
> Hi,
>
> I'm a little bit confused by all the memory limit options available in
> ActiveMQ, maybe someone could help me. I primarily send non-persistent
> messages, and I'd like them to be spooled to disk in case a global memory
> limit is reached.
>
> Basically, I'd like to set a global limit of 256MB for all queues together
> (not per queue) and I'd like to spool the non-persistent messages and the
> cursor to disk.
>
> At the moment, I am using the following configuration, but according to
> JConsole, it seems like the limits are not propagated correctly to the
> queues (e.g. JConsoles shows a memory limit of 256 on every queue) and I see
> heap-usage around 400-600MB at the moment.
>
>    <systemUsage>
>      <systemUsage sendFailIfNoSpace="true">
>        <memoryUsage>
>          <memoryUsage limit="256 mb" />
>        </memoryUsage>
>      </systemUsage>
>    </systemUsage>

The memoryUsage above should limit the entire broker to 256mb of
memory. I just tested this by lowering the memoryUsage limit to 15mb
and maxed that out pretty easily by flooding a queue with messages and
not consuming them very fast.

You can see the memory usage reported in the debug logging by enabling
debug logging via the conf/log4j.properties file as shown on the
following page:

http://activemq.apache.org/how-do-i-change-the-logging.html

Also, add the following logger to the conf/log4j.properties file:

log4j.logger.org.apache.activemq.usage.Usage=debug

This will ensure that you will get the debug output from the Usage
class which will output messages as the broker consumes more memory
like the following:

DEBUG | default:memory: usage change from: 29% of available memory,
to: 30% of available memory

However, if you are sending non-persistent messages, then you need to
set the tempUsage element within the systemUsage element. The
tempUsage element  governs the amount of disk space that is allowed to
be used to hold non-persistent messages.

>   <destinationPolicy>
>      <policyMap>
>        <policyEntries>
>          <policyEntry topic=">" producerFlowControl="false">
>          </policyEntry>
>          <policyEntry queue=">" producerFlowControl="false">
>            <deadLetterStrategy>
>              <individualDeadLetterStrategy queuePrefix="DLQ."
> useQueueForQueueMessages="true"
>                processExpired="true" processNonPersistent="false" />
>            </deadLetterStrategy>
>            <pendingQueuePolicy>
>              <fileQueueCursor />
>            </pendingQueuePolicy>
>          </policyEntry>
>        </policyEntries>
>      </policyMap>
>    </destinationPolicy>
>
> Thanks in advance for any pointer if this is correct or how I'd need to
> adapt this in order to achieve what I described above.

I also notice above that you have disabled producer flow control. By
disabling this setting, you have disallowed the broker from slowing
down producers who may be flooding the broker. For info see the
following:

http://activemq.apache.org/producer-flow-control.html

Also, what version of ActiveMQ are you using?

Bruce
-- 
perl -e 'print unpack("u30","D0G)U8V4\@4VYY9&5R\"F)R=6-E+G-N>61E<D\!G;6%I;\"YC;VT*"
);'

ActiveMQ in Action: http://bit.ly/2je6cQ
Blog: http://bruceblog.org/
Twitter: http://twitter.com/brucesnyder

Mime
View raw message