Return-Path: 
 <users-return-23964-apmail-activemq-users-archive=activemq.apache.org@activemq.apache.org>
Delivered-To: apmail-activemq-users-archive@www.apache.org
Received: (qmail 76820 invoked from network); 9 Jun 2010 16:51:43 -0000
Received: from unknown (HELO mail.apache.org) (140.211.11.3)
  by 140.211.11.9 with SMTP; 9 Jun 2010 16:51:43 -0000
Received: (qmail 81992 invoked by uid 500); 9 Jun 2010 16:51:43 -0000
Delivered-To: apmail-activemq-users-archive@activemq.apache.org
Received: (qmail 81962 invoked by uid 500); 9 Jun 2010 16:51:43 -0000
Mailing-List: contact users-help@activemq.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:users-help@activemq.apache.org>
List-Unsubscribe: <mailto:users-unsubscribe@activemq.apache.org>
List-Post: <mailto:users@activemq.apache.org>
List-Id: <users.activemq.apache.org>
Reply-To: users@activemq.apache.org
Delivered-To: mailing list users@activemq.apache.org
Received: (qmail 81954 invoked by uid 99); 9 Jun 2010 16:51:43 -0000
Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136)
    by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 09 Jun 2010 16:51:43 +0000
X-ASF-Spam-Status: No, hits=-1.2 required=10.0
	tests=AWL,RCVD_IN_DNSWL_LOW,SPF_NEUTRAL
X-Spam-Check-By: apache.org
Received-SPF: neutral (athena.apache.org: 206.15.76.4 is neither permitted nor
 denied by domain of dkrsmanovic@plos.org)
Received: from [206.15.76.4] (HELO mail1.plos.org) (206.15.76.4)
    by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 09 Jun 2010 16:51:37 +0000
Thread-Index: AcsH8/nxhvGbRYkpSRyEKuBBUUYxdw==
Content-Class: urn:content-classes:message
Importance: normal
Priority: normal
Received: from 10.135.2.141 ([10.135.2.141]) by SFEXCH01.plos.org
 ([10.135.1.7]) with Microsoft Exchange Server HTTP-DAV ;
 Wed,  9 Jun 2010 16:51:15 +0000
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.3790.4325
Received: from dkrsmanovic-usws by mail1.plos.org; 09 Jun 2010 09:51:21 -0700
Subject: Flow control, memory limits and persistent store confusion
From: "Dragisa Krsmanovic" <dkrsmanovic@plos.org>
To: <users@activemq.apache.org>
Content-Type: text/plain;
	charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Date: Wed, 09 Jun 2010 09:51:21 -0700
Message-ID: <1276102281.3144.33.camel@dkrsmanovic-usws>
MIME-Version: 1.0
X-Mailer: Evolution 2.28.3 

We've set up AMQ to persist all messages:

<broker xmlns=3D"http://activemq.apache.org/schema/core"
        brokerName=3D"localhost"
        useJmx=3D"true"
        persistent=3D"true">

    <persistenceAdapter>
      <amqPersistenceAdapter directory=3D"/foo"/>
    </persistenceAdapter>


We are using flow control for all queues (13 of them):
<policyEntry queue=3D">" producerFlowControl=3D"true" =
memoryLimit=3D"10mb"/>

System usage is set as follows:

  <systemUsage>
      <systemUsage>
        <memoryUsage>
          <memoryUsage limit=3D"180 mb"/>
        </memoryUsage>
        <storeUsage>
          <storeUsage limit=3D"3 gb" name=3D"plos-queue"/>
        </storeUsage>
        <tempUsage>
          <tempUsage limit=3D"100 mb"/>
        </tempUsage>
      </systemUsage>
    </systemUsage>

We have slow consumers and messages are sent in bursts of about 20,000.
I was expecting, since we persist all the messages, that I should be
able to put all 20k of them in the queue quickly and have consumers
slowly work on them.

Instead, it looks like none of the queues can have more than 140-150
messages at the time and flow control is making producer wait. It's like
queues are limited by the number of messages they can have in those 10m
of memory. Aren't persistent messages limited by store size ?

How do I achieve what I want ?=20

What would be a consequence in turning off flow control ?

Thanks

--=20
Dragisa Krsmanovic
Java Developer
Public Library of Science

-------------------------------------------------------------------------=
-------------------------------------------------------------------------=
------------------------------------------------------------------------
This email is confidential to the intended recipient. If you have =
received it in error, please notify the sender and delete it from your
system. Any unauthorized use, disclosure or copying is not permitted. =
The views or opinions presented are solely those of the sender and do
not necessarily represent those of Public Library of Science unless =
otherwise specifically stated. Please note that neither Public Library
of Science nor any of its agents accept any responsibility for any =
viruses that may be contained in this e-mail or its attachments and it
is your responsibility to scan the e-mail and attachments (if any).