activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joe Niski <>
Subject Re: producer flow control and message cursors
Date Tue, 05 Oct 2010 18:57:58 GMT
That's very helpful, Stan, i really appreciate the clarity and detail.  i'll follow up on the
systemUsage elements.

(re: the reply logic: sorry about that, it appears that one of our mail gateways is stripping
the reply-to header...)

Joe Niski
IS Development |  NWEA

PHONE 503.212.3382  |  FAX 503.639.7873

NWEA.ORG<3D%22>  |  Partnering to Help All Kids Learn

On 10/05/2010 11:42 AM, Stan Lewis wrote:
Sorry Joe, you want to reply to<>,
looks like
my usage of reply all confused things.

Looks like you want to increase the temp store system limit, not sure
off the top of my head what the default is, but it's probably similar
to the commented out setting in the default activemq.xml.  With these
settings I can put 1M messages onto a queue with no consumer using the
default 512MB heap limit:

                <policyEntry topic=">" producerFlowControl="true"
                <policyEntry queue=">" producerFlowControl="true"
 <!-- snip -->
                    <memoryUsage limit="20 mb"/>
                    <storeUsage limit="10 gb"/>
                    <tempUsage limit="10 gb"/>

When using the default store based cursor (and KahaDB for your
persistence adapter) the tempUsage limit tracks the size of
data/localhost/tmp_storage, though not sure if it just looks at the
one datafile or includes the whole directory, using the example
producer to push 1M messages gives me a temp store size of 5.5G.

In general flow control prevents some limit being reached, be it the
memory limit on a destination, the overall system memory limit or the
store/temp filesize limits, in a way that you wouldn't perhaps need to
address in your producer, i.e. if the limit is reached, your producer
blocks or slows down a bit, perhaps giving a consumer time to drain
the destination that the producer is sending messages to.  Message
cursors are another way to avoid high memory usage in the broker, by
spooling messages someplace other than memory (unless of course you're
using the vmcursor) as they're received.  In your case I suspect the
cursor is doing it's job but you have to increase the temp size limit,
as that's what kicks in when you're pushing non-persistent messages.
If you're pushing persistent messages then it'd be the store size

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message