activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Johan Edstrom <seij...@gmail.com>
Subject Re: Understanding memoryLimits with flow control and systemUsage limits.
Date Wed, 05 Jun 2013 15:39:29 GMT
Do it the other way around.
The system and store settings are global, the others per destination.
So what you are doing doesn't make a ton of sense.


On Jun 5, 2013, at 9:24 AM, Jesus Roncero <jesus@we7.com> wrote:

> Hi all,
> 
> I've got a testing activeMQ installation that I'm trying to test regarding
> different limits on destinations and memory usage in general, with and without
> flowControl enable. I'm running into some issues and I don't quite understand
> how this works, so I would appreciate any tips or advise here.
> 
> My setup is activeMQ 5.6 in a testing environment. I'm using a couple of
> python scripts that use stomp.py to send some messages and read them from a
> testqueue. Depending on a couple of settings, the behaviour changes, and it
> behaves differently.
> 
> I'm sendind 1 Mb messages to the queue as per:
> 
> data = "x" * 1024 * 1024
> number = 1000
> print "producing %s %s kb messages" % (number, len(data) / 1024)
> for i in range(number):
>    print i
>    conn.send(data, destination='/queue/testqueue')
> 
> in python.
> 
> Now, I have two settings:
> 
> Scenario number 1
> ----------------
> 
> <policyEntries>
>     <policyEntry queue=">" memoryLimit="50mb" producerFlowControl="true">
>         <pendingQueuePolicy>
>             <fileQueueCursor />
>         </pendingQueuePolicy>
>     </policyEntry>
>     <policyEntry topic=">" memoryLimit="50mb" producerFlowControl="true">
>         <!-- We want to store both persistent and non-persistent messages -->
>         <pendingSubscriberPolicy>
>             <fileCursor />
>         </pendingSubscriberPolicy>
>         <pendingDurableSubscriberPolicy>
>             <fileDurableSubscriberCursor/>
>         </pendingDurableSubscriberPolicy>
> 
>         ...
> 
>        <systemUsage>
>            <systemUsage>
>                <memoryUsage>
>                    <memoryUsage limit="40 mb"/>
>                </memoryUsage>
>                <storeUsage>
>                    <storeUsage limit="1 gb" />
>                </storeUsage>
>                <tempUsage>
>                    <tempUsage limit="800 mb"/>
>                </tempUsage>
>            </systemUsage>
>        </systemUsage>
> 
> Where the memoryLimit of each of the queues is bigger than the limit for
> memoryUsage. When I try to insert into the queue, it insert 32 messages (32Mb)
> and it hangs in there. I see this log messages:
> 
> [2013-06-05 15:54:19,816] INFO ActiveMQ Transport: tcp:///10.0.9.124:36431 org.apache.activemq.broker.region.Queue
- Usage Manager Memory Limit (52428800) reached on queue://testqueue. Producers will be throttled
to the rate at which messages are removed from this destination to prevent flooding it. See
http://activemq.apache.org/producer-flow-control.html for more info 
> 2013-06-05 15:54:19,816 [0.0.9.124:36431] INFO  Queue  - Usage Manager Memory Limit (52428800)
reached on queue://testqueue. Producers will be throttled to the rate at which messages are
removed from this destination to prevent flooding it. See http://activemq.apache.org/producer-flow-control.html
for more info 
> [2013-06-05 15:54:20,818] INFO ActiveMQ Transport: tcp:///10.0.9.124:36431 org.apache.activemq.broker.region.Queue
- Usage(default:memory:queue://testqueue:memory) percentUsage=62%, usage=32537724, limit=52428800,
percentUsageMinDelta=1%;Parent:Usage(default:memory) percentUsage=101%, usage=42443902, limit=41943040,
percentUsageMinDelta=1%: Usage Manager Memory Limit reached. Producer (ID:meme-49473-1370444050426-2:1:-1:1)
stopped to prevent flooding queue://testqueue. See http://activemq.apache.org/producer-flow-control.html
for more info (blocking for: 1s) 
> 2013-06-05 15:54:20,818 [0.0.9.124:36431] INFO  Queue  - Usage(default:memory:queue://testqueue:memory)
percentUsage=62%, usage=32537724, limit=52428800, percentUsageMinDelta=1%;Parent:Usage(default:memory)
percentUsage=101%, usage=42443902, limit=41943040, percentUsageMinDelta=1%: Usage Manager
Memory Limit reached. Producer (ID:meme-49473-1370444050426-2:1:-1:1) stopped to prevent flooding
queue://testqueue. See http://activemq.apache.org/producer-flow-control.html for more info
(blocking for: 1s) 
> 
> Which would indicate that flow control kicked in as per the first message,
> that they reach a memory limit (50Mb) as per the per destination limit (second
> message). It also says that the memory (40Mb) got used as a 100%.
> 
> Also, noted that it says: default:memory:queue.
> 
> 
> Scenario number 2
> -----------------
> 
> Now, if I change the memoryUsage to be slightly bigger than the memoryLimit on
> each queue, from 40 to 60Mb, as per:
> 
>        <systemUsage>
>            <systemUsage>
>                <memoryUsage>
>                    <memoryUsage limit="60 mb"/>
>                </memoryUsage>
>                <storeUsage>
>                    <storeUsage limit="1 gb" />
>                </storeUsage>
>                <tempUsage>
>                    <tempUsage limit="800 mb"/>
>                </tempUsage>
>            </systemUsage>
>        </systemUsage>
> 
> taking into account that the per-destination limits are just the same, 50Mb as
> per the config above. When I run the same exact procedure, it inserts 801 1Mb
> messages and it stops with this message:
> 
> [2013-06-05 16:05:01,972] INFO BrokerService.worker.1 org.apache.activemq.store.kahadb.plist.PListStore
- PListStore:[/var/lib/activemq/we7/data/localhost/tmp_storage ] initialized 
> 2013-06-05 16:05:01,972 [ervice.worker.1] INFO  PListStore - PListStore:[/var/lib/activemq/we7/data/localhost/tmp_storage
] initialized 
> [2013-06-05 16:05:12,959] INFO ActiveMQ Transport: tcp:///10.0.9.124:36615 org.apache.activemq.broker.region.Queue
- Usage(default:temp:queue://testqueue:temp) percentUsage=99%, usage=839122944, limit=838860800,
percentUsageMinDelta=1%;Parent:Usage(default:temp) percentUsage=100%, usage=839122944, limit=838860800,
percentUsageMinDelta=1%: Temp Store is Full (99% of 838860800). Stopping producer (ID:meme-55481-1370444692514-2:1:-1:1)
to prevent flooding queue://testqueue. See http://activemq.apache.org/producer-flow-control.html
for more info (blocking for: 1s) 
> 2013-06-05 16:05:12,959 [0.0.9.124:36615] INFO  Queue - Usage(default:temp:queue://testqueue:temp)
percentUsage=99%, usage=839122944, limit=838860800, percentUsageMinDelta=1%;Parent:Usage(default:temp)
percentUsage=100%, usage=839122944, limit=838860800, percentUsageMinDelta=1%: Temp Store is
Full (99% of 838860800). Stopping producer (ID:meme-55481-1370444692514-2:1:-1:1) to prevent
flooding queue://testqueue. See http://activemq.apache.org/producer-flow-control.html for
more info (blocking for: 1s) 
> 
> Where I see that it starts using the tempUsage space, and it pauses when it's
> used the 800 Mb available, whining that the temp store is full.
> 
> Now, the type is default:temp:queue rather than memory
> 
> So,
> 
> Could anyone explain to me why this is? I've read quite a few blog posts and
> entries on the mailing list and I'm still a bit confused about it, so, to have
> more concrete questions, 
> 
> 1) Why this change on the memoryUsage limit affects how activeMQ deals with
> way too many messages? More so, why on the second example, the 50 Mb limit
> doesn't kick in?
> 
> 2) I uderstood that activeMQ passes messages to the temp storage area when a
> queue is full 70% (unless configured otherwise), but this is not happening
> here. Why?
> 
> 3) I've seen a case where each queue is configured with a 500Mb limit,
> memoryUsage limit is 20Mb and tempUsage is 100Mb, activeMQ accepts 350 Mb
> worth of messages (70% of 500 Mb) and then it writes it the temporary storage,
> and then it hangs, complaining temp storage is 350% of the maximum value.
> Then, there's no way to recover activeMQ unless with a restart.
> 
> 
> Any ideas/suggestions/pointers would be appreciated, as I'm getting a bit
> frustrated trying to understand this :-)
> 
> Apologies for the length of the email, hope is easily understandable.
> 
> Thanks.
> Regards.
> 
> -- 
> JRF


Mime
View raw message