activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dragisa Krsmanovic" <dkrsmano...@plos.org>
Subject Flow control, memory limits and persistent store confusion
Date Wed, 09 Jun 2010 16:51:21 GMT
We've set up AMQ to persist all messages:

<broker xmlns="http://activemq.apache.org/schema/core"
        brokerName="localhost"
        useJmx="true"
        persistent="true">

    <persistenceAdapter>
      <amqPersistenceAdapter directory="/foo"/>
    </persistenceAdapter>


We are using flow control for all queues (13 of them):
<policyEntry queue=">" producerFlowControl="true" memoryLimit="10mb"/>

System usage is set as follows:

  <systemUsage>
      <systemUsage>
        <memoryUsage>
          <memoryUsage limit="180 mb"/>
        </memoryUsage>
        <storeUsage>
          <storeUsage limit="3 gb" name="plos-queue"/>
        </storeUsage>
        <tempUsage>
          <tempUsage limit="100 mb"/>
        </tempUsage>
      </systemUsage>
    </systemUsage>

We have slow consumers and messages are sent in bursts of about 20,000.
I was expecting, since we persist all the messages, that I should be
able to put all 20k of them in the queue quickly and have consumers
slowly work on them.

Instead, it looks like none of the queues can have more than 140-150
messages at the time and flow control is making producer wait. It's like
queues are limited by the number of messages they can have in those 10m
of memory. Aren't persistent messages limited by store size ?

How do I achieve what I want ? 

What would be a consequence in turning off flow control ?

Thanks

-- 
Dragisa Krsmanovic
Java Developer
Public Library of Science

--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
This email is confidential to the intended recipient. If you have received it in error, please
notify the sender and delete it from your
system. Any unauthorized use, disclosure or copying is not permitted. The views or opinions
presented are solely those of the sender and do
not necessarily represent those of Public Library of Science unless otherwise specifically
stated. Please note that neither Public Library
of Science nor any of its agents accept any responsibility for any viruses that may be contained
in this e-mail or its attachments and it
is your responsibility to scan the e-mail and attachments (if any).

Mime
View raw message