activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christian Posta <>
Subject Re: cursor memory usage/limit vs memory usage/limit
Date Mon, 29 Jul 2013 13:30:09 GMT
Give me one more piece of information.

What is the prefetch settings you're using for your topics (durable and
non-durable subscribers), and how many durable subscribers are connected?

On Sun, Jul 28, 2013 at 8:40 PM, Jake Choi <> wrote:

> Thanks a lot Christian for the well-explained documentation and this
> explains
> why I'm seeing the trouble with topic durable subscriptions regarding the
> per-destination memory usage & limit.
> First, let me quote some from your writing:
> "Main Broker Memory, Destination Memory, Subscription Memory
> ...
> A destination, when it’s created, will create its own SystemUsage object
> (which creates its own separate Memory, Store, and Temp Usage objects) but
> it will set its parent to the be broker’s main SystemUsage object. A
> destination can have its memory limits tuned individually (but not Store
> and
> Temp, those will still delegate to the parent). To set a destination’s
> memory limit:
> ...
> So the destination usage objects can be used to more finely control
> MemoryUsage, but it will always coordinate with the Main memory for all
> usage counts. This functionality can be used to limit the number of
> messages
> that a destination keeps around so that a single destination cannot starve
> other destinations. *For queues, it also affects the store cursor’s high
> water mark. A queue has different cursors for persistent and non-persistent
> messages. If we hit the high water mark (a threshold of the destination’s
> memory limit), no more messages be cached ready to be dispatched, and
> non-persistent messages can be purged to temp disk as necessary (if the
> StoreCursor will use FilePendingMessageCursor… otherwise it will just use a
> VMPendingMessageCursor and won’t purge to temporary store).*
> If you don’t specify a memory limit for individual destinations, the
> destination’s SystemUsage will delegate to the parent (Main SystemUsage)
> for
> all usage counts. This means it will effectively use the broker’s Main
> SystemUsage for all memory-related counts.
> *Consumer subscriptions, on the other hand, don’t have any notion of their
> own SystemUsage or MemoryUsage counters. They will always use the broker’s
> Main SystemUsage objects.* The main thing to note about this is when using
> a
> FilePendingMessageCursor for subscriptions (for example, for a Topic
> subscription), the messages will not be swapped to disk until the cursor
> high-water mark (70% by default) is reached.. but that means 70% of Main
> memory will need to be reached. That could be a while, and a lot of
> messages
> could be kept in memory. And if your subscription is the one holding most
> of
> those messages, swapping to disk could take a while. As topics dispatch
> messages to one subscription at a time, if one subscription grinds to a
> halt
> because it’s swapping its messages to disk, the rest of the subscription
> ready to receive the message will also feel the slow down..."
> What I don't understand from the above is why consumer subscriptions (and
> their cursors) of topics are not using per-destination MemoryLimit but
> share
> the Broker's main MemoryLimit, unlike the queues (please see the bolded
> sentences above).  Due to this + some logics inside AbstractStoreCursor's
> space checking logics, PFC always kicks in for the topic whenever the
> persistent store cursor's cache (pendingList) gets full.  Let me explain
> what's happening:
> 1. A topic is created, with per-destination memory limit = 1MB and broker's
> main memory limit = 5MB.
> 2. Topic's SystemUsage (for PFC purpose) is configured with per-destination
> memory limit (1MB), while TopicStorePrefetch(persistent store cursor)'s
> SystemUsage (for remaining cache space checking purpose) is configured with
> broker's main memory limit (10MB).
> 3. The 1st message of 0.8MB is published:
>    a. PFC doesn't kick in at Topic layer, as Topic#memoryUsage#isFull is
> true.
>    b. It's cached to the persistent cursor, as AbstractStoreCursor#hasSpace
> is true.
>    c. per-dest memory usage % becomes 90%, while main memory usage %
> becomes
> 9%.
> 4. The 2nd message of 0.8MB is published:
>    a. PFC doesn't kick in either, as per-dest memory usage is 90% (< 100%).
> This is fine.
>    b. *(I expect here that this second message shouldn't be cached to the
> cursor but invalidate it, but)this second message is also cached to the
> persistent cursor, as AbstractStoreCursor#hasSpace is still true: 9% < 70%
> (main memory usage high watermark)!!!*
>    c. per-dest memory usage % becomes 180%, while main memory usage %
> becomes 18%.
> 5. The 3rd message of 0.8MB is published:
>    a. *(I don't want this behavior but) PFC kicks in!!!* because
> per-destination memory limit is exceeded (180%).
> The behavior I wanted to see is: at #4.b. TopicStorePrefetch checks cursor
> memory availability against per-dest memory limit (not against broker's
> main
> memory limit) so that cursor pending cache gets disabled (just keeps what
> has been cached) w/o increasing the per-destination memory usage.
> I don't want to block persistent message publishing to topics by the
> per-destination memory limit but only for non-persistent message
> publishing,
> cause the store size allowed for a destination is relatively higher than
> the
> memory limit: e.g. I'd like to allow per-destination persistent message
> publishing up to 10GB disk space without being blocked by the memory limit
> &
> PFC...  How can I achieve this?
> --
> View this message in context:
> Sent from the ActiveMQ - User mailing list archive at

*Christian Posta*
twitter: @christianposta

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message