activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jake Choi ...@yahoo-inc.com>
Subject Re: cursor memory usage/limit vs memory usage/limit
Date Fri, 02 Aug 2013 16:42:49 GMT
Let me summarize my observations so far (and I think all of these are bugs in
activemq-core-5.7.0):

1. Improper PFC kicking-in to persistent topic message publishers
When a durable topic subscriber(s) is (are) not consuming, *persistent*
message publishing is blocked (PFC) by exceeding topic's *per-destination
memory limit*, not *store limit*.  In the given test code (
MemoryLimitTest.java
<http://activemq.2283324.n4.nabble.com/file/n4670061/MemoryLimitTest.java> 
), there were no non-durable subscribers connected hence *persistent*
message publishing should've gone through regardless of durable subscriber's
message consumption until the backlog size hits the total store limit.  The
given test code easily reproduces this problem with the following scenario:
   - Embedded broker is configured with 10mb main broker memory limit and
1mb per-destination memory limit.
   - Embedded broker is configured to use persistent:
BrokerService#isPersistent == true.
   - A durable topic subscriber subscribes with 0 prefetch limit.
   - A topic publisher starts publishing 0.4mb sized messages and gets
blocked after publishing 3 messages.

Per the code inspection, this is the side effect with the fact that the
store cursor of the topic durable subscriber (i.e. TopicStorePrefetch)
records its memory uses to the main broker's memory limit but not to the
per-destination memory limit.  

I verified that the scenario went through by replacing the main broker's
memory limit with the per-destination one for the store: persistent message
publishing checks the per-destination memory limit before being dispatched
to the cursor cache and if it's already over the high watermark, it disables
the cache and just gets only into the store w/o increasing the
per-destination memory limit hence will hardly reach the 100% of the
per-destination limit even afterwards.  Try toggling "patchApplied" to true
to test this in the given test code ( MemoryLimitTest.java
<http://activemq.2283324.n4.nabble.com/file/n4670061/MemoryLimitTest.java> 
).

2. There's no straightforward way to use VMPendingMessageCursor for
non-persistent topic message cursor, when using the
StoreDurableSubscriberCursor for the durable topic subscribers.

3. When exists, PendingDurableSubscriberPolicy is used more than once for
the pending durable subscriber instantiation so that the
StoreDurableSubscriberCursor lastly created and used doesn't have the
TopicStorePrefetch instance in it.

Please review all the above that I demonstrated and explained here and
possibly confirm.

Regards,
Jake



--
View this message in context: http://activemq.2283324.n4.nabble.com/cursor-memory-usage-limit-vs-memory-usage-limit-tp4669679p4670061.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Mime
View raw message