activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jake Choi>
Subject Re: cursor memory usage/limit vs memory usage/limit
Date Mon, 29 Jul 2013 00:40:15 GMT
Thanks a lot Christian for the well-explained documentation and this explains
why I'm seeing the trouble with topic durable subscriptions regarding the
per-destination memory usage & limit.

First, let me quote some from your writing:

"Main Broker Memory, Destination Memory, Subscription Memory


A destination, when it’s created, will create its own SystemUsage object
(which creates its own separate Memory, Store, and Temp Usage objects) but
it will set its parent to the be broker’s main SystemUsage object. A
destination can have its memory limits tuned individually (but not Store and
Temp, those will still delegate to the parent). To set a destination’s
memory limit:

So the destination usage objects can be used to more finely control
MemoryUsage, but it will always coordinate with the Main memory for all
usage counts. This functionality can be used to limit the number of messages
that a destination keeps around so that a single destination cannot starve
other destinations. *For queues, it also affects the store cursor’s high
water mark. A queue has different cursors for persistent and non-persistent
messages. If we hit the high water mark (a threshold of the destination’s
memory limit), no more messages be cached ready to be dispatched, and
non-persistent messages can be purged to temp disk as necessary (if the
StoreCursor will use FilePendingMessageCursor… otherwise it will just use a
VMPendingMessageCursor and won’t purge to temporary store).*

If you don’t specify a memory limit for individual destinations, the
destination’s SystemUsage will delegate to the parent (Main SystemUsage) for
all usage counts. This means it will effectively use the broker’s Main
SystemUsage for all memory-related counts.

*Consumer subscriptions, on the other hand, don’t have any notion of their
own SystemUsage or MemoryUsage counters. They will always use the broker’s
Main SystemUsage objects.* The main thing to note about this is when using a
FilePendingMessageCursor for subscriptions (for example, for a Topic
subscription), the messages will not be swapped to disk until the cursor
high-water mark (70% by default) is reached.. but that means 70% of Main
memory will need to be reached. That could be a while, and a lot of messages
could be kept in memory. And if your subscription is the one holding most of
those messages, swapping to disk could take a while. As topics dispatch
messages to one subscription at a time, if one subscription grinds to a halt
because it’s swapping its messages to disk, the rest of the subscription
ready to receive the message will also feel the slow down..."

What I don't understand from the above is why consumer subscriptions (and
their cursors) of topics are not using per-destination MemoryLimit but share
the Broker's main MemoryLimit, unlike the queues (please see the bolded
sentences above).  Due to this + some logics inside AbstractStoreCursor's
space checking logics, PFC always kicks in for the topic whenever the
persistent store cursor's cache (pendingList) gets full.  Let me explain
what's happening:

1. A topic is created, with per-destination memory limit = 1MB and broker's
main memory limit = 5MB.
2. Topic's SystemUsage (for PFC purpose) is configured with per-destination
memory limit (1MB), while TopicStorePrefetch(persistent store cursor)'s
SystemUsage (for remaining cache space checking purpose) is configured with
broker's main memory limit (10MB).
3. The 1st message of 0.8MB is published:
   a. PFC doesn't kick in at Topic layer, as Topic#memoryUsage#isFull is
   b. It's cached to the persistent cursor, as AbstractStoreCursor#hasSpace
is true.
   c. per-dest memory usage % becomes 90%, while main memory usage % becomes
4. The 2nd message of 0.8MB is published:
   a. PFC doesn't kick in either, as per-dest memory usage is 90% (< 100%).
This is fine.
   b. *(I expect here that this second message shouldn't be cached to the
cursor but invalidate it, but)this second message is also cached to the
persistent cursor, as AbstractStoreCursor#hasSpace is still true: 9% < 70%
(main memory usage high watermark)!!!*
   c. per-dest memory usage % becomes 180%, while main memory usage %
becomes 18%.
5. The 3rd message of 0.8MB is published:
   a. *(I don't want this behavior but) PFC kicks in!!!* because
per-destination memory limit is exceeded (180%).

The behavior I wanted to see is: at #4.b. TopicStorePrefetch checks cursor
memory availability against per-dest memory limit (not against broker's main
memory limit) so that cursor pending cache gets disabled (just keeps what
has been cached) w/o increasing the per-destination memory usage.

I don't want to block persistent message publishing to topics by the
per-destination memory limit but only for non-persistent message publishing,
cause the store size allowed for a destination is relatively higher than the
memory limit: e.g. I'd like to allow per-destination persistent message
publishing up to 10GB disk space without being blocked by the memory limit &
PFC...  How can I achieve this?

View this message in context:
Sent from the ActiveMQ - User mailing list archive at

View raw message