Mailing-List: contact users-help@qpid.apache.org; run by ezmlm
Precedence: bulk
Reply-To: users@qpid.apache.org
Received-SPF: pass (athena.apache.org: domain of gsim@redhat.com designates
 66.187.233.31 as permitted sender)
Message-ID: <49AFAF34.7030608@redhat.com>
Date: Thu, 05 Mar 2009 10:53:40 +0000
From: Gordon Sim <gsim@redhat.com>
Organization: Registered Address: Red Hat UK Ltd, Amberley Place, 107-111
 Peascod Street, Windsor, Berkshire, SI4 1TE, United Kingdom.,Registered in
 England and Wales under Company Registration No. 3798903,Directors: Michael
 Cunningham (USA), Charlie Peters (USA) and David Owens (Ireland)
User-Agent: Thunderbird 1.5.0.2 (X11/20060501)
MIME-Version: 1.0
To: users@qpid.apache.org
Subject: Re: Stress C++ broker without flow control
References: <3649D40ACFF240A2AAAA9FDCF07DAC81@HANNIE>
 <49ACFE24.9020400@redhat.com> <7FC16DC9D4C7484BB2796D4EDF093BC8@HANNIE>
 <49AD09A5.1090601@redhat.com> <49AD9721.6030105@redhat.com>
 <50B693524FBB4A58A6E0AC28C2FE5BE9@HANNIE>
In-Reply-To: <50B693524FBB4A58A6E0AC28C2FE5BE9@HANNIE>
Content-Type: text/plain; charset=Big5
Content-Transfer-Encoding: 7bit

Stephen wrote:
> Hi ,
> 
> I still have question on messages stored in C++ broker, but not releasing.
> 
> I am sending ~200k messages of 512bytes each from Java Client to C++ broker
> I found there is a limit in C++ broker of 100MB default of each queue, 
> when the limit reach,

You can alter this with the --default-queue-limit option; setting it to
0 turns off any default limit.


> 2009-mar-05 11:22:31 notice Journal "TplStore": Created
> 2009-mar-05 11:22:31 notice Store module initialized; 
> dir=/ebina/fno/qpid_c++_m4/qpid_store
> 2009-mar-05 11:22:31 notice SASL disabled: No Authentication Performed
> 2009-mar-05 11:22:31 notice Listening on TCP port 5678
> 2009-mar-05 11:22:31 notice Broker running
> 2009-mar-05 11:24:48 warning Message 204801 on 
> _@development_983f81f0-272a-48f9-aecd-0f203ebd3848 cannot be released 
> from memory as the queue is not durable
> 2009-mar-05 11:24:48 error Execution exception: resource-limit-exceeded: 
> Policy exceeded on _@development_983f81f0-272a-48f9-aecd-0f203ebd3848 by 
> message 204801 of size 512 , policy: size: max=104857600, 
> current=104857094; count: unlimited; type=flow_to_disk 
> (qpid/broker/QueuePolicy.cpp:90)
> 
> Then I shutdown the producer and consumer, And use qpid-tool to check 
> the exact queue which exceeded the limit
> 
>    Type       Element                113                 116 
> 119                 124              126
>    
> ======================================================================================================================== 
> 
>    property   vhostRef               103                 103 
> 103                 103              103
>    property   name                   _@development_b9e6 
> _@development_6d47b  _@development_9e79  mgmt-HXXXXXP0XX  repl-HXX
>    property   durable                False               False 
> False               False            False
>    property   autoDelete             False               False 
> False               True             True
>    property   exclusive              True                True 
> True                True             True
>    property   arguments              {}                  {} 
> {}                  {}               {}
>    statistic  msgTotalEnqueues       204801 messages     204800 
> 204797              653              45
>    statistic  msgTotalDequeues       204801              0 
> 0                   653              45
>    statistic  msgTxnEnqueues         0                   0 
> 0                   0                0
>    statistic  msgTxnDequeues         0                   0 
> 0                   0                0
>    statistic  msgPersistEnqueues     204801              204800 
> 204797              0                0
>    statistic  msgPersistDequeues     204801              0 
> 0                   0                0
>    statistic  msgDepth               0                   204800 
> 204797              0                0
>    statistic  byteDepth              0 octets            104857094 
> 104856064           0                0
>    statistic  byteTotalEnqueues      104857606           104857094 
> 104856064           86850            22238
>    statistic  byteTotalDequeues      104857606           0 
> 0                   86850            22238
>    statistic  byteTxnEnqueues        0                   0 
> 0                   0                0
>    statistic  byteTxnDequeues        0                   0 
> 0                   0                0
>    statistic  bytePersistEnqueues    104857606           104857094 
> 104856064           0                0
>    statistic  bytePersistDequeues    104857606           0 
> 0                   0                0
> 
> It seems that the messages are enqueue in those "queues" and never got 
> deleted. And (it seems that) these queues exist forever, without manual 
> actions.
> 
> Any idea / suggestion about the msg stored in memory / queues? I should 
> have "subscribed" / "consumed" them already
> 
> Here are my settings:
> Java producer is sending msg using url: 
> topic://development/admin_in?durable='false', with auto acknowledgement
> Java consumer is receiving msg from url: 
> topic://development/*.*?durable='false', with client acknowledgement
> 
> The binding and queue are created by C++ broker automatically.
> I have tried to increase the queue limit of C++ broker like this:
> 
> ./qpid-config -a guest/guest@127.0.0.1:5678 add exchange topic development
> ./qpid-config -a guest/guest@127.0.0.1:5678 add queue linux.news 
> --max-queue-size 209715200
> ./qpid-config -a guest/guest@127.0.0.1:5678 bind development linux.news
> 
> but no use, as the msgs are enqueued in other queues and do not dequeue.

A few questions:

Are you using durable subscriptions in JMS?
Are you issuing acknowledgements when in client acknowledged mode?
Are you using a specific client id?
What connection URL are you using for producer and consumer?

---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project:      http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org